Deep Learning Project¶

Flowers Images Classification¶

Davide Giardini - 897473

Import libraries¶

In [1]:
import torch
torch.cuda.is_available()
Out[1]:
True
In [2]:
# Import TensorFlow/Keras
import tensorflow as tf
from tensorflow import keras
# keras.__version__
from time import time
import numpy as np
import random as python_random
# Import other modules
from matplotlib import pyplot as plt
np.random.seed(0)
python_random.seed(0)
tf.random.set_seed(0)

Preprocessing¶

Evaluation of average image size¶

In [4]:
import cv2
import os
from tqdm import tqdm
In [4]:
base_path = 'C:/Users/david/Desktop/DeepLearning/flower_photos'
categories = os.listdir(base_path)
categories
Out[4]:
['daisy', 'dandelion', 'roses', 'sunflowers', 'Test', 'Train', 'tulips']
In [ ]:
height = []
width = []
for category in categories:
  for image in tqdm(os.listdir(base_path + category)):
    im = cv2.imread(base_path + category + '/' + image)
    height.append(im.shape[0])
    width.append(im.shape[1])

100%|██████████| 898/898 [00:47<00:00, 18.97it/s]
100%|██████████| 799/799 [00:28<00:00, 27.56it/s]
100%|██████████| 633/633 [00:21<00:00, 29.51it/s]
100%|██████████| 699/699 [00:24<00:00, 28.17it/s]
100%|██████████| 641/641 [00:29<00:00, 21.86it/s]
In [ ]:
# Setting up Seaborn style
sns.set(style="whitegrid")

# Creating horizontal boxplots
plt.figure(figsize=(5, 3))
g = sns.boxplot(data=[height, width], orient='h', palette='Set2')
plt.title('Horizontal Boxplot of Height and Width', fontsize=16)
plt.xticks(fontsize=12)
plt.yticks(fontsize=12)
g.set_yticklabels(['Height', 'Width'])
plt.grid(axis='x')
plt.tight_layout()
plt.show()

<ipython-input-33-60fba1ad0455>:10: UserWarning: FixedFormatter should only be used together with FixedLocator
  g.set_yticklabels(['Height', 'Width'])
In [ ]:
# Average width
sum(width)/len(width)

365.0623978201635
In [ ]:
# Average height
sum(height)/len(height)

271.79128065395093

Import data¶

We create two copies of the data, one named "Train" and one named "Test".

In [11]:
train_path = 'C:/Users/david/Desktop/DeepLearning/flower_photos/Train/'
In [5]:
train_dataset = keras.preprocessing.image_dataset_from_directory(train_path,
                                                            image_size=(256,256),
                                                            batch_size=32,
                                                            label_mode='categorical',
                                                            subset='training',
                                                            validation_split = 0.2,
                                                            shuffle=True,
                                                            seed=1)

Found 2936 files belonging to 5 classes.
Using 2349 files for training.
In [6]:
val_dataset = keras.preprocessing.image_dataset_from_directory(train_path,
                                                            image_size=(256,256),
                                                            batch_size=32,
                                                            label_mode='categorical',
                                                            subset='validation',
                                                            validation_split = 0.2,
                                                            shuffle=True,
                                                            seed=1)

Found 2936 files belonging to 5 classes.
Using 587 files for validation.

Split Train and Test folder¶

From the train data, we are going to remove all of the observations that have been selected in the validation dataset. In the same way, from the test data, we are going to remove all of the observations that have been selected in the train dataset.

In [ ]:
# From train dataset, remove all of the observations selected in validation set
for el in val_dataset.file_paths:
  os.remove(el)
In [ ]:
import re
In [ ]:
# From test dataset, remove all of the observations selected in train set
for el in train_dataset.file_paths:
  # Substitute 'Train' with 'Test' in order to remove observation from train directory
  el = re.sub('Train', 'Test', el)
  os.remove(el)

A look at the data¶

In [1]:
import os
train_path = 'C:/Users/david/Desktop/DeepLearning/flower_photos/Train/'
subD = os.listdir(train_path)
dizTrain = {}
# For each class subdirectory, insert in the dictionary dizTrain the number of element in that subdirectory
for dir in subD:
    dizTrain[dir] = len(os.listdir(train_path + dir))
In [2]:
# Number of observations of each class in train set
dizTrain
Out[2]:
{'daisy': 508,
 'dandelion': 701,
 'roses': 534,
 'sunflowers': 561,
 'tulips': 632}
In [3]:
test_path = 'C:/Users/david/Desktop/DeepLearning/flower_photos/Test/'
subD = os.listdir(test_path)
dizTest = {}
# For each class subdirectory, insert in the dictionary dizTrain the number of element in that subdirectory
for dir in subD:
    dizTest[dir] = len(os.listdir(test_path + dir))
In [4]:
# Number of observations of each class in train set
dizTest
Out[4]:
{'daisy': 125,
 'dandelion': 197,
 'roses': 107,
 'sunflowers': 138,
 'tulips': 167}
In [8]:
plt.bar(dizTrain.keys(), dizTrain.values(), color='b', alpha=0.5, label='Train')
plt.bar(dizTest.keys(), dizTest.values(), color='r', alpha=0.5, label='Test')

# Adding labels and title
plt.xlabel('N. of Observations')
plt.ylabel('Classes')
plt.title('N. of Observations per class')
plt.legend()

# Show plot
plt.show()

As we can see, there is not a huge difference in the classes sizes. For now, we are going to try and develop CNNs that do not address the issues of unbalanced classes.

Import data and split between train and Validation¶

The train data is going to be split in 80% train set and 20% validation set. The test data is going to be loaded from the train set that we built previously.
To recap: we firstly split the entire dataset in 80% train and 20% test. Now, we split the train set in 80% train and 20% validation. In this way, we have 0.64% train + 0.16% validation + 20% test.

In [12]:
train_path = 'C:/Users/david/Desktop/DeepLearning/flower_photos/Train/'
test_path = 'C:/Users/david/Desktop/DeepLearning/flower_photos/Test/'
In [8]:
train_ds = keras.preprocessing.image_dataset_from_directory(train_path,
                                                            image_size=(224,224),
                                                            batch_size=32,
                                                            label_mode='categorical',
                                                            subset='training',
                                                            validation_split = 0.2,
                                                            shuffle=True,
                                                            seed=1)

Found 2936 files belonging to 5 classes.
Using 2349 files for training.
In [9]:
val_ds = keras.preprocessing.image_dataset_from_directory(train_path,
                                                            image_size=(224,224),
                                                            batch_size=32,
                                                            label_mode='categorical',
                                                            subset='validation',
                                                            validation_split = 0.2,
                                                            shuffle=True,
                                                            seed=1)

Found 2936 files belonging to 5 classes.
Using 587 files for validation.
In [10]:
test_ds = keras.preprocessing.image_dataset_from_directory(test_path,
                                                            image_size=(224,224),
                                                            batch_size=32,
                                                            label_mode='categorical',
                                                            shuffle=False,
                                                            seed=1)

Found 734 files belonging to 5 classes.
In [15]:
num_classes = 5

First Models¶

We are going to start with a very easy CNN, and see what changes benefit its performances.

The first CNN consists of two hidden layers, with 32 and 64 neurons respectively. Both layers with ReLU activation.
Max Pooling is applied to the first layer, and Global Max Pooling to the second.

In [45]:
# Architecture definition Naive Model
inputs = keras.Input((224,224,3))

x = inputs
x = keras.layers.Conv2D(32, 3, padding="same")(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.MaxPooling2D(3, strides=3, padding="same")(x)

x = keras.layers.Conv2D(64, 3, padding="same")(x)
x = keras.layers.Activation("relu")(x)

x = keras.layers.GlobalMaxPooling2D()(x)

outputs = keras.layers.Dense(num_classes, activation="softmax")(x)
net = keras.Model(inputs, outputs)
In [15]:
# Compile the model for training
net.compile(loss=keras.losses.categorical_crossentropy,
            optimizer=keras.optimizers.RMSprop(learning_rate=0.001),
            metrics=['accuracy'])
In [16]:
# Check that we are running on GPU
tf.config.list_physical_devices('GPU')
Out[16]:
[PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]
In [17]:
# Model training
history = net.fit(train_ds,
          epochs=40,
          validation_data=val_ds)

Epoch 1/40
74/74 [==============================] - 11s 72ms/step - loss: 17.0196 - accuracy: 0.3274 - val_loss: 5.2585 - val_accuracy: 0.4242
Epoch 2/40
74/74 [==============================] - 4s 52ms/step - loss: 4.1190 - accuracy: 0.4551 - val_loss: 4.1449 - val_accuracy: 0.3986
Epoch 3/40
74/74 [==============================] - 4s 50ms/step - loss: 2.0812 - accuracy: 0.5143 - val_loss: 2.0645 - val_accuracy: 0.5026
Epoch 4/40
74/74 [==============================] - 4s 51ms/step - loss: 1.4421 - accuracy: 0.5734 - val_loss: 1.5696 - val_accuracy: 0.5298
Epoch 5/40
74/74 [==============================] - 4s 50ms/step - loss: 1.1769 - accuracy: 0.6228 - val_loss: 1.9620 - val_accuracy: 0.5094
Epoch 6/40
74/74 [==============================] - 4s 52ms/step - loss: 0.9984 - accuracy: 0.6624 - val_loss: 1.4287 - val_accuracy: 0.5554
Epoch 7/40
74/74 [==============================] - 4s 52ms/step - loss: 0.8333 - accuracy: 0.7092 - val_loss: 1.4772 - val_accuracy: 0.5622
Epoch 8/40
74/74 [==============================] - 5s 61ms/step - loss: 0.7523 - accuracy: 0.7399 - val_loss: 1.7338 - val_accuracy: 0.5332
Epoch 9/40
74/74 [==============================] - 5s 64ms/step - loss: 0.6737 - accuracy: 0.7603 - val_loss: 1.5059 - val_accuracy: 0.5963
Epoch 10/40
74/74 [==============================] - 5s 68ms/step - loss: 0.6115 - accuracy: 0.7897 - val_loss: 1.2401 - val_accuracy: 0.6440
Epoch 11/40
74/74 [==============================] - 5s 69ms/step - loss: 0.5533 - accuracy: 0.7991 - val_loss: 1.5348 - val_accuracy: 0.6116
Epoch 12/40
74/74 [==============================] - 5s 66ms/step - loss: 0.4892 - accuracy: 0.8191 - val_loss: 1.3223 - val_accuracy: 0.5843
Epoch 13/40
74/74 [==============================] - 5s 67ms/step - loss: 0.4653 - accuracy: 0.8357 - val_loss: 1.2628 - val_accuracy: 0.6031
Epoch 14/40
74/74 [==============================] - 5s 66ms/step - loss: 0.4180 - accuracy: 0.8540 - val_loss: 1.5855 - val_accuracy: 0.6048
Epoch 15/40
74/74 [==============================] - 5s 67ms/step - loss: 0.3965 - accuracy: 0.8608 - val_loss: 1.2333 - val_accuracy: 0.6525
Epoch 16/40
74/74 [==============================] - 5s 67ms/step - loss: 0.3618 - accuracy: 0.8782 - val_loss: 1.5441 - val_accuracy: 0.6031
Epoch 17/40
74/74 [==============================] - 5s 66ms/step - loss: 0.3258 - accuracy: 0.8868 - val_loss: 2.0238 - val_accuracy: 0.6014
Epoch 18/40
74/74 [==============================] - 5s 66ms/step - loss: 0.3315 - accuracy: 0.8817 - val_loss: 1.3188 - val_accuracy: 0.6337
Epoch 19/40
74/74 [==============================] - 5s 72ms/step - loss: 0.2685 - accuracy: 0.9076 - val_loss: 1.3513 - val_accuracy: 0.6167
Epoch 20/40
74/74 [==============================] - 6s 74ms/step - loss: 0.2843 - accuracy: 0.8983 - val_loss: 1.4823 - val_accuracy: 0.6422
Epoch 21/40
74/74 [==============================] - 6s 74ms/step - loss: 0.2466 - accuracy: 0.9153 - val_loss: 1.5257 - val_accuracy: 0.6644
Epoch 22/40
74/74 [==============================] - 5s 69ms/step - loss: 0.2286 - accuracy: 0.9281 - val_loss: 1.5254 - val_accuracy: 0.6508
Epoch 23/40
74/74 [==============================] - 5s 69ms/step - loss: 0.2380 - accuracy: 0.9221 - val_loss: 1.2851 - val_accuracy: 0.6729
Epoch 24/40
74/74 [==============================] - 5s 69ms/step - loss: 0.2063 - accuracy: 0.9327 - val_loss: 2.9742 - val_accuracy: 0.5094
Epoch 25/40
74/74 [==============================] - 5s 67ms/step - loss: 0.2015 - accuracy: 0.9319 - val_loss: 1.3905 - val_accuracy: 0.6695
Epoch 26/40
74/74 [==============================] - 5s 66ms/step - loss: 0.1916 - accuracy: 0.9417 - val_loss: 1.6560 - val_accuracy: 0.6201
Epoch 27/40
74/74 [==============================] - 5s 66ms/step - loss: 0.1841 - accuracy: 0.9438 - val_loss: 2.1389 - val_accuracy: 0.6031
Epoch 28/40
74/74 [==============================] - 5s 66ms/step - loss: 0.1963 - accuracy: 0.9383 - val_loss: 1.7148 - val_accuracy: 0.6405
Epoch 29/40
74/74 [==============================] - 5s 66ms/step - loss: 0.1410 - accuracy: 0.9502 - val_loss: 1.4132 - val_accuracy: 0.6712
Epoch 30/40
74/74 [==============================] - 5s 66ms/step - loss: 0.1609 - accuracy: 0.9527 - val_loss: 2.4283 - val_accuracy: 0.6150
Epoch 31/40
74/74 [==============================] - 5s 66ms/step - loss: 0.1597 - accuracy: 0.9523 - val_loss: 1.8876 - val_accuracy: 0.6286
Epoch 32/40
74/74 [==============================] - 5s 66ms/step - loss: 0.1274 - accuracy: 0.9549 - val_loss: 1.9109 - val_accuracy: 0.6576
Epoch 33/40
74/74 [==============================] - 5s 66ms/step - loss: 0.1617 - accuracy: 0.9472 - val_loss: 1.7023 - val_accuracy: 0.6491
Epoch 34/40
74/74 [==============================] - 5s 68ms/step - loss: 0.1564 - accuracy: 0.9527 - val_loss: 1.5465 - val_accuracy: 0.6559
Epoch 35/40
74/74 [==============================] - 5s 67ms/step - loss: 0.1210 - accuracy: 0.9672 - val_loss: 1.7433 - val_accuracy: 0.6695
Epoch 36/40
74/74 [==============================] - 5s 66ms/step - loss: 0.1734 - accuracy: 0.9510 - val_loss: 1.5100 - val_accuracy: 0.6848
Epoch 37/40
74/74 [==============================] - 5s 66ms/step - loss: 0.1139 - accuracy: 0.9642 - val_loss: 2.8824 - val_accuracy: 0.5945
Epoch 38/40
74/74 [==============================] - 5s 66ms/step - loss: 0.1256 - accuracy: 0.9579 - val_loss: 1.8765 - val_accuracy: 0.6627
Epoch 39/40
74/74 [==============================] - 5s 67ms/step - loss: 0.1353 - accuracy: 0.9587 - val_loss: 2.0762 - val_accuracy: 0.6320
Epoch 40/40
74/74 [==============================] - 5s 67ms/step - loss: 0.0931 - accuracy: 0.9719 - val_loss: 2.7361 - val_accuracy: 0.5554
In [19]:
# Visualization of the learning curves

plt.subplot(1, 2, 1)
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.ylim([0.0, 2.0])
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.legend(['train', 'valid'])

plt.subplot(1, 2, 2)
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.ylim([0.5, 1.0])
plt.xlabel('Epoch')
plt.ylabel('Accuracy')
plt.legend(['train', 'valid'])
Out[19]:
<matplotlib.legend.Legend at 0x2a4a520c6d0>

While this simple Convolutional Neural Network reaches +95% accuracy on the training set, it is evident that it does not generalize well on the validation set. We will have to find additional methods to close the gap between train accuracy and validation accuracy.

Batch Normalization¶

The process of batch normalization makes it so that the weight in the network do not become unbalanced. We are going to add Batch Normalization on both of the hidden layers of the network.

In [20]:
# Architecture definition
inputs = keras.Input((224,224,3))

x = inputs
x = keras.layers.Conv2D(32, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.MaxPooling2D(3, strides=3, padding="same")(x)

x = keras.layers.Conv2D(64, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)

x = keras.layers.GlobalMaxPooling2D()(x)

outputs = keras.layers.Dense(num_classes, activation="softmax")(x)
net = keras.Model(inputs, outputs)
In [21]:
# Compile the model for training
net.compile(loss=keras.losses.categorical_crossentropy,
            optimizer=keras.optimizers.RMSprop(learning_rate=0.001),
            metrics=['accuracy'])
In [22]:
# Model training (v2)
history = net.fit(train_ds,
          epochs=40,
          validation_data=val_ds);

Epoch 1/40
74/74 [==============================] - 8s 83ms/step - loss: 1.2796 - accuracy: 0.5121 - val_loss: 2.8535 - val_accuracy: 0.3424
Epoch 2/40
74/74 [==============================] - 6s 74ms/step - loss: 0.9084 - accuracy: 0.6390 - val_loss: 1.2577 - val_accuracy: 0.4940
Epoch 3/40
74/74 [==============================] - 6s 80ms/step - loss: 0.7982 - accuracy: 0.6845 - val_loss: 1.0072 - val_accuracy: 0.5894
Epoch 4/40
74/74 [==============================] - 6s 87ms/step - loss: 0.7335 - accuracy: 0.7143 - val_loss: 0.7672 - val_accuracy: 0.7070
Epoch 5/40
74/74 [==============================] - 7s 88ms/step - loss: 0.6749 - accuracy: 0.7361 - val_loss: 0.8029 - val_accuracy: 0.6780
Epoch 6/40
74/74 [==============================] - 7s 87ms/step - loss: 0.6239 - accuracy: 0.7680 - val_loss: 0.7348 - val_accuracy: 0.7206
Epoch 7/40
74/74 [==============================] - 7s 88ms/step - loss: 0.5900 - accuracy: 0.7842 - val_loss: 0.8029 - val_accuracy: 0.6848
Epoch 8/40
74/74 [==============================] - 7s 87ms/step - loss: 0.5679 - accuracy: 0.7808 - val_loss: 0.8248 - val_accuracy: 0.6865
Epoch 9/40
74/74 [==============================] - 7s 88ms/step - loss: 0.5212 - accuracy: 0.8118 - val_loss: 0.8332 - val_accuracy: 0.6831
Epoch 10/40
74/74 [==============================] - 7s 88ms/step - loss: 0.5138 - accuracy: 0.8131 - val_loss: 0.7168 - val_accuracy: 0.7257
Epoch 11/40
74/74 [==============================] - 7s 88ms/step - loss: 0.4825 - accuracy: 0.8280 - val_loss: 0.7173 - val_accuracy: 0.7274
Epoch 12/40
74/74 [==============================] - 7s 88ms/step - loss: 0.4600 - accuracy: 0.8318 - val_loss: 0.7322 - val_accuracy: 0.7206
Epoch 13/40
74/74 [==============================] - 7s 88ms/step - loss: 0.4364 - accuracy: 0.8378 - val_loss: 0.8179 - val_accuracy: 0.6848
Epoch 14/40
74/74 [==============================] - 7s 88ms/step - loss: 0.4334 - accuracy: 0.8463 - val_loss: 0.7783 - val_accuracy: 0.6951
Epoch 15/40
74/74 [==============================] - 7s 88ms/step - loss: 0.4077 - accuracy: 0.8582 - val_loss: 0.6688 - val_accuracy: 0.7581
Epoch 16/40
74/74 [==============================] - 7s 87ms/step - loss: 0.3873 - accuracy: 0.8740 - val_loss: 0.8979 - val_accuracy: 0.6729
Epoch 17/40
74/74 [==============================] - 7s 96ms/step - loss: 0.3656 - accuracy: 0.8838 - val_loss: 0.7228 - val_accuracy: 0.7070
Epoch 18/40
74/74 [==============================] - 8s 105ms/step - loss: 0.3494 - accuracy: 0.8834 - val_loss: 0.7341 - val_accuracy: 0.7155
Epoch 19/40
74/74 [==============================] - 9s 122ms/step - loss: 0.3515 - accuracy: 0.8838 - val_loss: 0.6989 - val_accuracy: 0.7223
Epoch 20/40
74/74 [==============================] - 10s 133ms/step - loss: 0.3113 - accuracy: 0.9012 - val_loss: 0.8656 - val_accuracy: 0.6865
Epoch 21/40
74/74 [==============================] - 10s 138ms/step - loss: 0.3170 - accuracy: 0.9046 - val_loss: 1.2112 - val_accuracy: 0.6133
Epoch 22/40
74/74 [==============================] - 12s 156ms/step - loss: 0.3106 - accuracy: 0.8991 - val_loss: 0.6845 - val_accuracy: 0.7376
Epoch 23/40
74/74 [==============================] - 12s 158ms/step - loss: 0.2756 - accuracy: 0.9191 - val_loss: 0.8177 - val_accuracy: 0.7121
Epoch 24/40
74/74 [==============================] - 13s 175ms/step - loss: 0.2774 - accuracy: 0.9174 - val_loss: 1.0374 - val_accuracy: 0.6167
Epoch 25/40
74/74 [==============================] - 14s 194ms/step - loss: 0.2549 - accuracy: 0.9212 - val_loss: 0.7430 - val_accuracy: 0.7138
Epoch 26/40
74/74 [==============================] - 15s 202ms/step - loss: 0.2540 - accuracy: 0.9264 - val_loss: 0.8174 - val_accuracy: 0.6865
Epoch 27/40
74/74 [==============================] - 15s 205ms/step - loss: 0.2570 - accuracy: 0.9144 - val_loss: 0.7751 - val_accuracy: 0.7155
Epoch 28/40
74/74 [==============================] - 15s 205ms/step - loss: 0.2361 - accuracy: 0.9293 - val_loss: 0.9054 - val_accuracy: 0.6712
Epoch 29/40
74/74 [==============================] - 15s 206ms/step - loss: 0.2183 - accuracy: 0.9434 - val_loss: 0.8155 - val_accuracy: 0.7189
Epoch 30/40
74/74 [==============================] - 16s 208ms/step - loss: 0.2175 - accuracy: 0.9353 - val_loss: 0.7847 - val_accuracy: 0.7240
Epoch 31/40
74/74 [==============================] - 16s 208ms/step - loss: 0.2085 - accuracy: 0.9468 - val_loss: 0.7389 - val_accuracy: 0.7325
Epoch 32/40
74/74 [==============================] - 16s 209ms/step - loss: 0.1941 - accuracy: 0.9485 - val_loss: 0.9924 - val_accuracy: 0.6865
Epoch 33/40
74/74 [==============================] - 16s 216ms/step - loss: 0.1850 - accuracy: 0.9476 - val_loss: 0.8727 - val_accuracy: 0.7104
Epoch 34/40
74/74 [==============================] - 16s 215ms/step - loss: 0.1784 - accuracy: 0.9579 - val_loss: 0.9229 - val_accuracy: 0.6814
Epoch 35/40
74/74 [==============================] - 16s 216ms/step - loss: 0.1794 - accuracy: 0.9476 - val_loss: 0.8788 - val_accuracy: 0.7342
Epoch 36/40
74/74 [==============================] - 16s 216ms/step - loss: 0.1597 - accuracy: 0.9621 - val_loss: 0.7931 - val_accuracy: 0.7428
Epoch 37/40
74/74 [==============================] - 17s 226ms/step - loss: 0.1594 - accuracy: 0.9587 - val_loss: 1.1626 - val_accuracy: 0.6542
Epoch 38/40
74/74 [==============================] - 17s 225ms/step - loss: 0.1509 - accuracy: 0.9647 - val_loss: 0.8755 - val_accuracy: 0.6917
Epoch 39/40
74/74 [==============================] - 18s 230ms/step - loss: 0.1513 - accuracy: 0.9651 - val_loss: 0.7679 - val_accuracy: 0.7496
Epoch 40/40
74/74 [==============================] - 18s 240ms/step - loss: 0.1355 - accuracy: 0.9672 - val_loss: 0.8849 - val_accuracy: 0.6882
In [51]:
# Visualization of the learning curves

plt.subplot(1, 2, 1)
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.ylim([0.0, 2.0])
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.legend(['train', 'valid'])

plt.subplot(1, 2, 2)
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.ylim([0.5, 1.0])
plt.xlabel('Epoch')
plt.ylabel('Accuracy')
plt.legend(['train', 'valid'])
Out[51]:
<matplotlib.legend.Legend at 0x29101c77160>

It seems that adding batch Normalization results in an increase in performance.
Though, there is a lot of volatility, and the network is still not able to generalize as well as we would want. In order to try and reduce this gap, we are going to add Data Augmentation.

Data Augmentation¶

Data Augmentation consists in artificially augmenting the training data by, in this case, applying transformations over the observations. This will not only increase the size of the training data, but will increase its complexity. This increase in complexity will likely prevent overfitting, but will come at the cost of worse performances on the training data. This is not a problem, since, in case this happen, we can resolve the issue be increasing the complexity of our Neural Network.

Since we want to apply a "crop" augmentation, we have to re-import the training that as larger images, in order to later crop them into the default 224x224 size that we used for the previous CNNs.

In [11]:
# Data loader with (slightly) larger images
# In this way, we can augment the images with RandomCrop
train_ds_256 = keras.preprocessing.image_dataset_from_directory(train_path,
                                                            image_size=(256,256),
                                                            batch_size=32,
                                                            label_mode='categorical',
                                                            subset='training',
                                                            validation_split = 0.2,
                                                            shuffle=True,
                                                            seed=1)

Found 2936 files belonging to 5 classes.
Using 2349 files for training.
In [11]:
# We check if the file did not change. This should be the case since we inserted the same seed.
train_ds.file_paths == train_ds_256.file_paths
Out[11]:
True

For augmenting our data, we apply three functions: RandomCrop, RandomFlip and RandomContrast. Note: these 3 types of augmentation have been decided via testing different combinations, for shortness and clarity purposes I'm not going to display all of them.
For what regards everything else, the architechture will remain exactly the same. In this way we are going to test the improvements that this technique brings.

In [ ]:
# Architecture definition
inputs = keras.Input((None,None,3))

x = inputs

x = keras.layers.RandomCrop(224, 224)(x)
x = keras.layers.RandomFlip(mode='horizontal')(x)
x = keras.layers.RandomContrast(0.25)(x)

x = keras.layers.Conv2D(32, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.MaxPooling2D(3, strides=3, padding="same")(x)

x = keras.layers.Conv2D(64, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)

x = keras.layers.GlobalMaxPooling2D()(x)

outputs = keras.layers.Dense(num_classes, activation="softmax")(x)
net = keras.Model(inputs, outputs)
In [ ]:
# Compile the model for training
net.compile(loss=keras.losses.categorical_crossentropy,
            optimizer=keras.optimizers.RMSprop(learning_rate=0.001),
            metrics=['accuracy'])
In [ ]:
# Model training
history = net.fit(train_ds_256,
          epochs=40,
          validation_data=val_ds)

Epoch 1/40
74/74 [==============================] - 8s 96ms/step - loss: 1.3436 - accuracy: 0.4874 - val_loss: 1.7722 - val_accuracy: 0.3952
Epoch 2/40
74/74 [==============================] - 6s 82ms/step - loss: 0.9842 - accuracy: 0.6113 - val_loss: 1.0545 - val_accuracy: 0.5809
Epoch 3/40
74/74 [==============================] - 6s 85ms/step - loss: 0.8703 - accuracy: 0.6552 - val_loss: 0.8448 - val_accuracy: 0.6627
Epoch 4/40
74/74 [==============================] - 8s 102ms/step - loss: 0.8052 - accuracy: 0.6901 - val_loss: 0.9303 - val_accuracy: 0.6440
Epoch 5/40
74/74 [==============================] - 8s 102ms/step - loss: 0.7661 - accuracy: 0.7016 - val_loss: 0.8709 - val_accuracy: 0.6678
Epoch 6/40
74/74 [==============================] - 8s 102ms/step - loss: 0.7624 - accuracy: 0.7143 - val_loss: 0.7269 - val_accuracy: 0.7223
Epoch 7/40
74/74 [==============================] - 8s 102ms/step - loss: 0.6959 - accuracy: 0.7318 - val_loss: 0.7989 - val_accuracy: 0.6934
Epoch 8/40
74/74 [==============================] - 8s 102ms/step - loss: 0.6920 - accuracy: 0.7361 - val_loss: 0.9533 - val_accuracy: 0.6354
Epoch 9/40
74/74 [==============================] - 8s 102ms/step - loss: 0.6681 - accuracy: 0.7484 - val_loss: 0.8612 - val_accuracy: 0.6934
Epoch 10/40
74/74 [==============================] - 8s 102ms/step - loss: 0.6730 - accuracy: 0.7314 - val_loss: 0.7826 - val_accuracy: 0.6985
Epoch 11/40
74/74 [==============================] - 8s 102ms/step - loss: 0.6420 - accuracy: 0.7544 - val_loss: 0.8309 - val_accuracy: 0.6797
Epoch 12/40
74/74 [==============================] - 8s 102ms/step - loss: 0.6308 - accuracy: 0.7718 - val_loss: 0.6283 - val_accuracy: 0.7513
Epoch 13/40
74/74 [==============================] - 8s 102ms/step - loss: 0.6210 - accuracy: 0.7599 - val_loss: 0.6973 - val_accuracy: 0.7308
Epoch 14/40
74/74 [==============================] - 8s 102ms/step - loss: 0.6098 - accuracy: 0.7654 - val_loss: 0.9147 - val_accuracy: 0.6508
Epoch 15/40
74/74 [==============================] - 8s 105ms/step - loss: 0.5968 - accuracy: 0.7722 - val_loss: 0.5948 - val_accuracy: 0.7513
Epoch 16/40
74/74 [==============================] - 8s 106ms/step - loss: 0.5622 - accuracy: 0.7914 - val_loss: 0.6408 - val_accuracy: 0.7496
Epoch 17/40
74/74 [==============================] - 8s 105ms/step - loss: 0.5561 - accuracy: 0.7871 - val_loss: 0.7929 - val_accuracy: 0.6985
Epoch 18/40
74/74 [==============================] - 9s 120ms/step - loss: 0.5691 - accuracy: 0.7876 - val_loss: 0.6081 - val_accuracy: 0.7598
Epoch 19/40
74/74 [==============================] - 10s 140ms/step - loss: 0.5526 - accuracy: 0.7940 - val_loss: 0.6719 - val_accuracy: 0.7496
Epoch 20/40
74/74 [==============================] - 11s 145ms/step - loss: 0.5376 - accuracy: 0.8012 - val_loss: 0.6843 - val_accuracy: 0.7376
Epoch 21/40
74/74 [==============================] - 12s 162ms/step - loss: 0.5292 - accuracy: 0.7969 - val_loss: 0.8255 - val_accuracy: 0.7121
Epoch 22/40
74/74 [==============================] - 12s 161ms/step - loss: 0.5283 - accuracy: 0.7995 - val_loss: 0.7709 - val_accuracy: 0.6865
Epoch 23/40
74/74 [==============================] - 13s 178ms/step - loss: 0.5315 - accuracy: 0.7995 - val_loss: 0.6507 - val_accuracy: 0.7581
Epoch 24/40
74/74 [==============================] - 14s 188ms/step - loss: 0.5148 - accuracy: 0.8110 - val_loss: 0.5927 - val_accuracy: 0.7853
Epoch 25/40
74/74 [==============================] - 14s 191ms/step - loss: 0.5048 - accuracy: 0.8072 - val_loss: 0.7018 - val_accuracy: 0.7445
Epoch 26/40
74/74 [==============================] - 15s 198ms/step - loss: 0.5118 - accuracy: 0.8050 - val_loss: 0.6363 - val_accuracy: 0.7411
Epoch 27/40
74/74 [==============================] - 14s 191ms/step - loss: 0.4918 - accuracy: 0.8191 - val_loss: 0.7743 - val_accuracy: 0.7291
Epoch 28/40
74/74 [==============================] - 15s 205ms/step - loss: 0.4896 - accuracy: 0.8165 - val_loss: 0.5867 - val_accuracy: 0.7683
Epoch 29/40
74/74 [==============================] - 14s 195ms/step - loss: 0.4812 - accuracy: 0.8221 - val_loss: 0.6388 - val_accuracy: 0.7547
Epoch 30/40
74/74 [==============================] - 15s 204ms/step - loss: 0.4766 - accuracy: 0.8199 - val_loss: 0.5497 - val_accuracy: 0.7973
Epoch 31/40
74/74 [==============================] - 16s 215ms/step - loss: 0.4924 - accuracy: 0.8169 - val_loss: 0.5546 - val_accuracy: 0.7888
Epoch 32/40
74/74 [==============================] - 16s 217ms/step - loss: 0.4519 - accuracy: 0.8357 - val_loss: 0.9878 - val_accuracy: 0.6559
Epoch 33/40
74/74 [==============================] - 16s 221ms/step - loss: 0.4612 - accuracy: 0.8272 - val_loss: 0.5623 - val_accuracy: 0.7683
Epoch 34/40
74/74 [==============================] - 16s 218ms/step - loss: 0.4693 - accuracy: 0.8272 - val_loss: 0.6548 - val_accuracy: 0.7513
Epoch 35/40
74/74 [==============================] - 16s 221ms/step - loss: 0.4385 - accuracy: 0.8357 - val_loss: 0.5771 - val_accuracy: 0.7905
Epoch 36/40
74/74 [==============================] - 17s 228ms/step - loss: 0.4331 - accuracy: 0.8433 - val_loss: 0.5939 - val_accuracy: 0.7802
Epoch 37/40
74/74 [==============================] - 17s 231ms/step - loss: 0.4391 - accuracy: 0.8429 - val_loss: 0.5964 - val_accuracy: 0.7615
Epoch 38/40
74/74 [==============================] - 17s 228ms/step - loss: 0.4496 - accuracy: 0.8272 - val_loss: 0.5499 - val_accuracy: 0.7956
Epoch 39/40
74/74 [==============================] - 17s 231ms/step - loss: 0.4413 - accuracy: 0.8387 - val_loss: 0.6639 - val_accuracy: 0.7359
Epoch 40/40
74/74 [==============================] - 17s 225ms/step - loss: 0.4262 - accuracy: 0.8421 - val_loss: 0.6685 - val_accuracy: 0.7445
In [ ]:
# Visualization of the learning curves

plt.subplot(1, 2, 1)
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.ylim([0.0, 2.0])
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.legend(['train', 'valid'])

plt.subplot(1, 2, 2)
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.ylim([0.5, 1.0])
plt.xlabel('Epoch')
plt.ylabel('Accuracy')
plt.legend(['train', 'valid'])

<matplotlib.legend.Legend at 0x26f0b1a3ac0>

Data augmentation did lower the accuracy on the training set, partially resolving the overfitting problem of the model. Though, we have a really high volatilty on the validation set, we are going to try to address this by lowering the Learning Rate.

Lower Learning Rate¶

In [55]:
inputs = keras.Input((None,None,3))

x = inputs

x = keras.layers.RandomCrop(224, 224)(x)
x = keras.layers.RandomFlip(mode='horizontal')(x)
x = keras.layers.RandomContrast(0.25)(x)

x = keras.layers.Conv2D(32, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.MaxPooling2D(3, strides=3, padding="same")(x)

x = keras.layers.Conv2D(64, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)

x = keras.layers.GlobalMaxPooling2D()(x)

outputs = keras.layers.Dense(num_classes, activation="softmax")(x)
net = keras.Model(inputs, outputs)
In [56]:
# Compile the model for training
net.compile(loss=keras.losses.categorical_crossentropy,
            optimizer=keras.optimizers.RMSprop(learning_rate=0.0005),
            metrics=['accuracy'])
In [57]:
# Model training (v4)
history = net.fit(train_ds_256,
          epochs=40,
          validation_data=val_ds)

Epoch 1/40
74/74 [==============================] - 9s 102ms/step - loss: 1.5953 - accuracy: 0.4444 - val_loss: 2.8384 - val_accuracy: 0.3203
Epoch 2/40
74/74 [==============================] - 6s 80ms/step - loss: 1.0027 - accuracy: 0.6075 - val_loss: 1.7423 - val_accuracy: 0.4838
Epoch 3/40
74/74 [==============================] - 6s 82ms/step - loss: 0.8753 - accuracy: 0.6603 - val_loss: 0.9845 - val_accuracy: 0.6218
Epoch 4/40
74/74 [==============================] - 7s 87ms/step - loss: 0.8201 - accuracy: 0.6833 - val_loss: 0.9869 - val_accuracy: 0.6235
Epoch 5/40
74/74 [==============================] - 7s 99ms/step - loss: 0.7881 - accuracy: 0.7016 - val_loss: 0.7551 - val_accuracy: 0.7121
Epoch 6/40
74/74 [==============================] - 8s 102ms/step - loss: 0.7534 - accuracy: 0.7156 - val_loss: 0.7740 - val_accuracy: 0.6712
Epoch 7/40
74/74 [==============================] - 8s 102ms/step - loss: 0.7294 - accuracy: 0.7160 - val_loss: 0.7080 - val_accuracy: 0.7189
Epoch 8/40
74/74 [==============================] - 8s 102ms/step - loss: 0.7141 - accuracy: 0.7216 - val_loss: 0.8438 - val_accuracy: 0.6491
Epoch 9/40
74/74 [==============================] - 8s 102ms/step - loss: 0.6819 - accuracy: 0.7395 - val_loss: 0.7652 - val_accuracy: 0.7019
Epoch 10/40
74/74 [==============================] - 8s 103ms/step - loss: 0.6797 - accuracy: 0.7386 - val_loss: 0.7234 - val_accuracy: 0.6985
Epoch 11/40
74/74 [==============================] - 8s 104ms/step - loss: 0.6761 - accuracy: 0.7420 - val_loss: 0.7929 - val_accuracy: 0.6917
Epoch 12/40
74/74 [==============================] - 8s 103ms/step - loss: 0.6376 - accuracy: 0.7544 - val_loss: 0.6648 - val_accuracy: 0.7445
Epoch 13/40
74/74 [==============================] - 8s 102ms/step - loss: 0.6376 - accuracy: 0.7607 - val_loss: 0.6231 - val_accuracy: 0.7462
Epoch 14/40
74/74 [==============================] - 8s 102ms/step - loss: 0.6321 - accuracy: 0.7629 - val_loss: 0.7515 - val_accuracy: 0.6934
Epoch 15/40
74/74 [==============================] - 8s 102ms/step - loss: 0.6138 - accuracy: 0.7808 - val_loss: 0.6523 - val_accuracy: 0.7325
Epoch 16/40
74/74 [==============================] - 8s 103ms/step - loss: 0.5874 - accuracy: 0.7854 - val_loss: 0.7071 - val_accuracy: 0.7206
Epoch 17/40
74/74 [==============================] - 8s 103ms/step - loss: 0.5853 - accuracy: 0.7816 - val_loss: 0.6338 - val_accuracy: 0.7479
Epoch 18/40
74/74 [==============================] - 8s 103ms/step - loss: 0.5944 - accuracy: 0.7782 - val_loss: 0.6311 - val_accuracy: 0.7359
Epoch 19/40
74/74 [==============================] - 8s 106ms/step - loss: 0.5701 - accuracy: 0.7880 - val_loss: 0.7424 - val_accuracy: 0.6917
Epoch 20/40
74/74 [==============================] - 8s 111ms/step - loss: 0.5681 - accuracy: 0.7914 - val_loss: 0.6108 - val_accuracy: 0.7717
Epoch 21/40
74/74 [==============================] - 8s 110ms/step - loss: 0.5590 - accuracy: 0.7842 - val_loss: 0.7066 - val_accuracy: 0.7325
Epoch 22/40
74/74 [==============================] - 9s 116ms/step - loss: 0.5591 - accuracy: 0.7854 - val_loss: 0.6848 - val_accuracy: 0.7240
Epoch 23/40
74/74 [==============================] - 8s 111ms/step - loss: 0.5510 - accuracy: 0.7897 - val_loss: 0.6353 - val_accuracy: 0.7394
Epoch 24/40
74/74 [==============================] - 10s 136ms/step - loss: 0.5267 - accuracy: 0.8063 - val_loss: 0.6195 - val_accuracy: 0.7666
Epoch 25/40
74/74 [==============================] - 14s 192ms/step - loss: 0.5175 - accuracy: 0.8029 - val_loss: 0.6502 - val_accuracy: 0.7462
Epoch 26/40
74/74 [==============================] - 15s 203ms/step - loss: 0.5375 - accuracy: 0.7978 - val_loss: 0.6433 - val_accuracy: 0.7411
Epoch 27/40
74/74 [==============================] - 17s 232ms/step - loss: 0.5148 - accuracy: 0.8157 - val_loss: 0.7694 - val_accuracy: 0.6985
Epoch 28/40
74/74 [==============================] - 18s 239ms/step - loss: 0.5121 - accuracy: 0.8118 - val_loss: 0.7182 - val_accuracy: 0.7155
Epoch 29/40
74/74 [==============================] - 18s 240ms/step - loss: 0.5051 - accuracy: 0.8093 - val_loss: 0.5768 - val_accuracy: 0.7785
Epoch 30/40
74/74 [==============================] - 18s 234ms/step - loss: 0.4833 - accuracy: 0.8242 - val_loss: 0.6223 - val_accuracy: 0.7734
Epoch 31/40
74/74 [==============================] - 17s 234ms/step - loss: 0.4980 - accuracy: 0.8186 - val_loss: 0.6950 - val_accuracy: 0.7342
Epoch 32/40
74/74 [==============================] - 17s 233ms/step - loss: 0.4957 - accuracy: 0.8272 - val_loss: 0.5990 - val_accuracy: 0.7700
Epoch 33/40
74/74 [==============================] - 15s 204ms/step - loss: 0.4892 - accuracy: 0.8186 - val_loss: 0.5941 - val_accuracy: 0.7802
Epoch 34/40
74/74 [==============================] - 14s 188ms/step - loss: 0.4904 - accuracy: 0.8225 - val_loss: 0.7020 - val_accuracy: 0.7223
Epoch 35/40
74/74 [==============================] - 17s 227ms/step - loss: 0.4722 - accuracy: 0.8318 - val_loss: 0.5980 - val_accuracy: 0.7581
Epoch 36/40
74/74 [==============================] - 17s 232ms/step - loss: 0.4696 - accuracy: 0.8335 - val_loss: 0.5836 - val_accuracy: 0.7888
Epoch 37/40
74/74 [==============================] - 17s 232ms/step - loss: 0.4704 - accuracy: 0.8297 - val_loss: 0.6748 - val_accuracy: 0.7445
Epoch 38/40
74/74 [==============================] - 17s 234ms/step - loss: 0.4576 - accuracy: 0.8352 - val_loss: 0.5817 - val_accuracy: 0.7905
Epoch 39/40
74/74 [==============================] - 17s 234ms/step - loss: 0.4481 - accuracy: 0.8276 - val_loss: 0.5982 - val_accuracy: 0.7649
Epoch 40/40
74/74 [==============================] - 18s 236ms/step - loss: 0.4517 - accuracy: 0.8438 - val_loss: 0.6717 - val_accuracy: 0.7291
In [58]:
# Visualization of the learning curves

plt.subplot(1, 2, 1)
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.ylim([0.0, 2.0])
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.legend(['train', 'valid'])

plt.subplot(1, 2, 2)
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.ylim([0.5, 1.0])
plt.xlabel('Epoch')
plt.ylabel('Accuracy')
plt.legend(['train', 'valid'])
Out[58]:
<matplotlib.legend.Legend at 0x29190aacfd0>

The volatility in the validation set did decrease a little, though, it is still present. Another possible cause of this could be that we chose a validation set that is too small. By choosing a small validation set, it could be possible that, at each epoch, the changes in the weights of the model that occur influence the classification of a big portion of the observation. We are going to try to see if, by increasing the validation set to a 30%, the validation accuracy is going to result more stable.

Change validation-set size from 20% to 30%¶

The train data is going to be split in 70% train set and 30% validation set. The test data is not going to be touched.
To recap: we firstly split the entire dataset in 80% train and 20% test. Now, we split the train set in 70% train and 30% validation. In this way, we have 0.56% train + 0.24% validation + 20% test.

In [4]:
train_ds = keras.preprocessing.image_dataset_from_directory(train_path,
                                                            image_size=(224,224),
                                                            batch_size=32,
                                                            label_mode='categorical',
                                                            subset='training',
                                                            validation_split = 0.3,
                                                            shuffle=True,
                                                            seed=1)

val_ds = keras.preprocessing.image_dataset_from_directory(train_path,
                                                            image_size=(224,224),
                                                            batch_size=32,
                                                            label_mode='categorical',
                                                            subset='validation',
                                                            validation_split = 0.3,
                                                            shuffle=True,
                                                            seed=1)

# Data loader with (slightly) larger images
# In this way, we can augment the images with RandomCrop
train_ds_256 = keras.preprocessing.image_dataset_from_directory(train_path,
                                                            image_size=(256,256),
                                                            batch_size=32,
                                                            label_mode='categorical',
                                                            subset='training',
                                                            validation_split = 0.3,
                                                            shuffle=True,
                                                            seed=1)

Found 2936 files belonging to 5 classes.
Using 2056 files for training.
Found 2936 files belonging to 5 classes.
Using 880 files for validation.
Found 2936 files belonging to 5 classes.
Using 2056 files for training.
In [ ]:
# We check if the file did not change. This should be the case since we inserted the same seed.
train_ds.file_paths == train_ds_256.file_paths

True

We are now going to retrain the exact same model, in order to evaluate the effects that this change in train/validation proportion have caused.

In [68]:
inputs = keras.Input((None,None,3))

x = inputs

x = keras.layers.RandomCrop(224, 224)(x)
x = keras.layers.RandomFlip(mode='horizontal')(x)
x = keras.layers.RandomContrast(0.25)(x)

x = keras.layers.Conv2D(32, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.MaxPooling2D(3, strides=3, padding="same")(x)

x = keras.layers.Conv2D(64, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)

x = keras.layers.GlobalMaxPooling2D()(x)

outputs = keras.layers.Dense(num_classes, activation="softmax")(x)
net = keras.Model(inputs, outputs)
In [69]:
# Compile the model for training
net.compile(loss=keras.losses.categorical_crossentropy,
            optimizer=keras.optimizers.RMSprop(learning_rate=0.0005),
            metrics=['accuracy'])
In [70]:
# Model training (v4)
history = net.fit(train_ds_256,
          epochs=40,
          validation_data=val_ds)

Epoch 1/40
65/65 [==============================] - 7s 87ms/step - loss: 1.3042 - accuracy: 0.4762 - val_loss: 2.0753 - val_accuracy: 0.2500
Epoch 2/40
65/65 [==============================] - 5s 83ms/step - loss: 1.0225 - accuracy: 0.5963 - val_loss: 1.6847 - val_accuracy: 0.3591
Epoch 3/40
65/65 [==============================] - 5s 81ms/step - loss: 0.9078 - accuracy: 0.6552 - val_loss: 1.0748 - val_accuracy: 0.5943
Epoch 4/40
65/65 [==============================] - 5s 83ms/step - loss: 0.8515 - accuracy: 0.6727 - val_loss: 0.9327 - val_accuracy: 0.6557
Epoch 5/40
65/65 [==============================] - 5s 82ms/step - loss: 0.8149 - accuracy: 0.6916 - val_loss: 0.7812 - val_accuracy: 0.7125
Epoch 6/40
65/65 [==============================] - 5s 82ms/step - loss: 0.8000 - accuracy: 0.6960 - val_loss: 0.7368 - val_accuracy: 0.7239
Epoch 7/40
65/65 [==============================] - 5s 83ms/step - loss: 0.7586 - accuracy: 0.6994 - val_loss: 0.7413 - val_accuracy: 0.7193
Epoch 8/40
65/65 [==============================] - 6s 90ms/step - loss: 0.7401 - accuracy: 0.7155 - val_loss: 0.7022 - val_accuracy: 0.7409
Epoch 9/40
65/65 [==============================] - 6s 99ms/step - loss: 0.7204 - accuracy: 0.7281 - val_loss: 0.6799 - val_accuracy: 0.7545
Epoch 10/40
65/65 [==============================] - 7s 102ms/step - loss: 0.6974 - accuracy: 0.7451 - val_loss: 0.6902 - val_accuracy: 0.7420
Epoch 11/40
65/65 [==============================] - 7s 102ms/step - loss: 0.6932 - accuracy: 0.7417 - val_loss: 0.6731 - val_accuracy: 0.7330
Epoch 12/40
65/65 [==============================] - 7s 104ms/step - loss: 0.6616 - accuracy: 0.7544 - val_loss: 0.6477 - val_accuracy: 0.7648
Epoch 13/40
65/65 [==============================] - 7s 105ms/step - loss: 0.6597 - accuracy: 0.7539 - val_loss: 0.6650 - val_accuracy: 0.7523
Epoch 14/40
65/65 [==============================] - 7s 104ms/step - loss: 0.6553 - accuracy: 0.7510 - val_loss: 0.6314 - val_accuracy: 0.7705
Epoch 15/40
65/65 [==============================] - 7s 103ms/step - loss: 0.6380 - accuracy: 0.7631 - val_loss: 0.7422 - val_accuracy: 0.7114
Epoch 16/40
65/65 [==============================] - 7s 103ms/step - loss: 0.6191 - accuracy: 0.7685 - val_loss: 0.6797 - val_accuracy: 0.7489
Epoch 17/40
65/65 [==============================] - 7s 104ms/step - loss: 0.6023 - accuracy: 0.7729 - val_loss: 0.6411 - val_accuracy: 0.7670
Epoch 18/40
65/65 [==============================] - 7s 105ms/step - loss: 0.6151 - accuracy: 0.7631 - val_loss: 0.6419 - val_accuracy: 0.7648
Epoch 19/40
65/65 [==============================] - 7s 104ms/step - loss: 0.5948 - accuracy: 0.7840 - val_loss: 0.6076 - val_accuracy: 0.7784
Epoch 20/40
65/65 [==============================] - 7s 106ms/step - loss: 0.5941 - accuracy: 0.7855 - val_loss: 0.6179 - val_accuracy: 0.7761
Epoch 21/40
65/65 [==============================] - 7s 105ms/step - loss: 0.5789 - accuracy: 0.7875 - val_loss: 0.5913 - val_accuracy: 0.7886
Epoch 22/40
65/65 [==============================] - 7s 106ms/step - loss: 0.5659 - accuracy: 0.7928 - val_loss: 0.6034 - val_accuracy: 0.7920
Epoch 23/40
65/65 [==============================] - 7s 105ms/step - loss: 0.5554 - accuracy: 0.7967 - val_loss: 0.6032 - val_accuracy: 0.7773
Epoch 24/40
65/65 [==============================] - 7s 106ms/step - loss: 0.5604 - accuracy: 0.7840 - val_loss: 0.6507 - val_accuracy: 0.7545
Epoch 25/40
65/65 [==============================] - 7s 106ms/step - loss: 0.5399 - accuracy: 0.8040 - val_loss: 0.5865 - val_accuracy: 0.7716
Epoch 26/40
65/65 [==============================] - 7s 106ms/step - loss: 0.5455 - accuracy: 0.8084 - val_loss: 0.6499 - val_accuracy: 0.7670
Epoch 27/40
65/65 [==============================] - 7s 105ms/step - loss: 0.5299 - accuracy: 0.7972 - val_loss: 0.6414 - val_accuracy: 0.7636
Epoch 28/40
65/65 [==============================] - 7s 111ms/step - loss: 0.5370 - accuracy: 0.7899 - val_loss: 0.6361 - val_accuracy: 0.7705
Epoch 29/40
65/65 [==============================] - 8s 120ms/step - loss: 0.5387 - accuracy: 0.8006 - val_loss: 0.6129 - val_accuracy: 0.7670
Epoch 30/40
65/65 [==============================] - 9s 139ms/step - loss: 0.5332 - accuracy: 0.8084 - val_loss: 0.6121 - val_accuracy: 0.7648
Epoch 31/40
65/65 [==============================] - 9s 139ms/step - loss: 0.5283 - accuracy: 0.8093 - val_loss: 0.7042 - val_accuracy: 0.7375
Epoch 32/40
65/65 [==============================] - 10s 155ms/step - loss: 0.5090 - accuracy: 0.8181 - val_loss: 0.6109 - val_accuracy: 0.7773
Epoch 33/40
65/65 [==============================] - 10s 158ms/step - loss: 0.5148 - accuracy: 0.8205 - val_loss: 0.5606 - val_accuracy: 0.7977
Epoch 34/40
65/65 [==============================] - 10s 160ms/step - loss: 0.5220 - accuracy: 0.8093 - val_loss: 0.5806 - val_accuracy: 0.8000
Epoch 35/40
65/65 [==============================] - 10s 151ms/step - loss: 0.5034 - accuracy: 0.8074 - val_loss: 0.6464 - val_accuracy: 0.7466
Epoch 36/40
65/65 [==============================] - 12s 184ms/step - loss: 0.4980 - accuracy: 0.8152 - val_loss: 0.6571 - val_accuracy: 0.7648
Epoch 37/40
65/65 [==============================] - 12s 185ms/step - loss: 0.4958 - accuracy: 0.8230 - val_loss: 0.5518 - val_accuracy: 0.8034
Epoch 38/40
65/65 [==============================] - 12s 188ms/step - loss: 0.4931 - accuracy: 0.8166 - val_loss: 0.5952 - val_accuracy: 0.7750
Epoch 39/40
65/65 [==============================] - 13s 207ms/step - loss: 0.4759 - accuracy: 0.8220 - val_loss: 0.6290 - val_accuracy: 0.7932
Epoch 40/40
65/65 [==============================] - 14s 215ms/step - loss: 0.4899 - accuracy: 0.8157 - val_loss: 0.5530 - val_accuracy: 0.7898
In [71]:
# Visualization of the learning curves

plt.subplot(1, 2, 1)
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.ylim([0.0, 2.0])
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.legend(['train', 'valid'])

plt.subplot(1, 2, 2)
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.ylim([0.5, 1.0])
plt.xlabel('Epoch')
plt.ylabel('Accuracy')
plt.legend(['train', 'valid'])
Out[71]:
<matplotlib.legend.Legend at 0x1cc18b1d610>

The validation accuracy seems more stable than before.

In conclusion: as we expected, data augmentation partially resolves the problem of over fitting, and achieves better validation accuracy. This of course came at the cost of lower performances on the training set. We will tackle this issue by increasing the model complexity.

Deeper CNN¶

As a first thing, we are going to add a third hidden layer of 128 neurons.

In [59]:
# Architecture definition
inputs = keras.Input((None,None,3))

x = inputs

x = keras.layers.RandomCrop(224, 224)(x)
x = keras.layers.RandomFlip(mode='horizontal')(x)
x = keras.layers.RandomContrast(0.25)(x)

x = keras.layers.Conv2D(32, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.MaxPooling2D(3, strides=3, padding="same")(x)

x = keras.layers.Conv2D(64, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.MaxPooling2D(3, strides=3, padding="same")(x)

x = keras.layers.Conv2D(128, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)

x = keras.layers.GlobalMaxPooling2D()(x)

outputs = keras.layers.Dense(num_classes, activation="softmax")(x)
net = keras.Model(inputs, outputs)
In [60]:
# Compile the model for training
net.compile(loss=keras.losses.categorical_crossentropy,
            optimizer=keras.optimizers.RMSprop(learning_rate=0.0005),
            metrics=['accuracy'])
In [61]:
# Model training
history = net.fit(train_ds_256,
          epochs=40,
          validation_data=val_ds)

Epoch 1/40
74/74 [==============================] - 9s 104ms/step - loss: 1.1801 - accuracy: 0.5615 - val_loss: 2.9854 - val_accuracy: 0.3101
Epoch 2/40
74/74 [==============================] - 8s 107ms/step - loss: 0.8641 - accuracy: 0.6662 - val_loss: 1.1181 - val_accuracy: 0.5537
Epoch 3/40
74/74 [==============================] - 9s 112ms/step - loss: 0.7560 - accuracy: 0.7071 - val_loss: 1.0258 - val_accuracy: 0.6167
Epoch 4/40
74/74 [==============================] - 8s 110ms/step - loss: 0.6986 - accuracy: 0.7335 - val_loss: 0.8233 - val_accuracy: 0.6780
Epoch 5/40
74/74 [==============================] - 8s 109ms/step - loss: 0.6259 - accuracy: 0.7548 - val_loss: 0.9193 - val_accuracy: 0.6968
Epoch 6/40
74/74 [==============================] - 8s 105ms/step - loss: 0.5955 - accuracy: 0.7850 - val_loss: 0.7864 - val_accuracy: 0.7138
Epoch 7/40
74/74 [==============================] - 8s 105ms/step - loss: 0.5705 - accuracy: 0.7914 - val_loss: 0.6738 - val_accuracy: 0.7376
Epoch 8/40
74/74 [==============================] - 8s 106ms/step - loss: 0.5586 - accuracy: 0.7884 - val_loss: 0.6647 - val_accuracy: 0.7530
Epoch 9/40
74/74 [==============================] - 8s 105ms/step - loss: 0.5033 - accuracy: 0.8165 - val_loss: 0.7908 - val_accuracy: 0.7138
Epoch 10/40
74/74 [==============================] - 8s 106ms/step - loss: 0.4942 - accuracy: 0.8131 - val_loss: 0.6498 - val_accuracy: 0.7394
Epoch 11/40
74/74 [==============================] - 8s 106ms/step - loss: 0.4764 - accuracy: 0.8208 - val_loss: 0.6490 - val_accuracy: 0.7888
Epoch 12/40
74/74 [==============================] - 8s 106ms/step - loss: 0.4443 - accuracy: 0.8306 - val_loss: 0.6542 - val_accuracy: 0.7496
Epoch 13/40
74/74 [==============================] - 8s 109ms/step - loss: 0.4315 - accuracy: 0.8370 - val_loss: 0.9572 - val_accuracy: 0.6542
Epoch 14/40
74/74 [==============================] - 9s 127ms/step - loss: 0.4178 - accuracy: 0.8467 - val_loss: 0.6553 - val_accuracy: 0.7768
Epoch 15/40
74/74 [==============================] - 10s 135ms/step - loss: 0.4019 - accuracy: 0.8570 - val_loss: 0.6063 - val_accuracy: 0.7853
Epoch 16/40
74/74 [==============================] - 11s 143ms/step - loss: 0.3753 - accuracy: 0.8616 - val_loss: 0.5881 - val_accuracy: 0.7751
Epoch 17/40
74/74 [==============================] - 11s 147ms/step - loss: 0.3722 - accuracy: 0.8693 - val_loss: 0.5423 - val_accuracy: 0.7888
Epoch 18/40
74/74 [==============================] - 11s 152ms/step - loss: 0.3640 - accuracy: 0.8778 - val_loss: 0.6124 - val_accuracy: 0.7598
Epoch 19/40
74/74 [==============================] - 12s 166ms/step - loss: 0.3582 - accuracy: 0.8770 - val_loss: 0.5808 - val_accuracy: 0.7922
Epoch 20/40
74/74 [==============================] - 12s 160ms/step - loss: 0.3224 - accuracy: 0.8799 - val_loss: 0.5237 - val_accuracy: 0.8177
Epoch 21/40
74/74 [==============================] - 12s 162ms/step - loss: 0.3214 - accuracy: 0.8851 - val_loss: 0.5256 - val_accuracy: 0.8092
Epoch 22/40
74/74 [==============================] - 13s 180ms/step - loss: 0.3229 - accuracy: 0.8872 - val_loss: 0.5818 - val_accuracy: 0.7751
Epoch 23/40
74/74 [==============================] - 13s 177ms/step - loss: 0.3120 - accuracy: 0.8936 - val_loss: 0.5717 - val_accuracy: 0.7785
Epoch 24/40
74/74 [==============================] - 12s 166ms/step - loss: 0.2927 - accuracy: 0.8948 - val_loss: 0.5214 - val_accuracy: 0.8126
Epoch 25/40
74/74 [==============================] - 14s 184ms/step - loss: 0.2820 - accuracy: 0.9017 - val_loss: 0.4979 - val_accuracy: 0.8313
Epoch 26/40
74/74 [==============================] - 14s 189ms/step - loss: 0.2875 - accuracy: 0.9034 - val_loss: 0.5901 - val_accuracy: 0.7649
Epoch 27/40
74/74 [==============================] - 15s 201ms/step - loss: 0.2771 - accuracy: 0.9012 - val_loss: 0.6517 - val_accuracy: 0.7700
Epoch 28/40
74/74 [==============================] - 17s 228ms/step - loss: 0.2645 - accuracy: 0.9072 - val_loss: 0.5557 - val_accuracy: 0.7785
Epoch 29/40
74/74 [==============================] - 16s 211ms/step - loss: 0.2541 - accuracy: 0.9170 - val_loss: 0.5317 - val_accuracy: 0.7871
Epoch 30/40
74/74 [==============================] - 15s 206ms/step - loss: 0.2428 - accuracy: 0.9212 - val_loss: 0.4973 - val_accuracy: 0.8143
Epoch 31/40
74/74 [==============================] - 14s 195ms/step - loss: 0.2525 - accuracy: 0.9127 - val_loss: 0.6951 - val_accuracy: 0.7700
Epoch 32/40
74/74 [==============================] - 15s 197ms/step - loss: 0.2427 - accuracy: 0.9127 - val_loss: 0.5116 - val_accuracy: 0.8143
Epoch 33/40
74/74 [==============================] - 15s 201ms/step - loss: 0.2192 - accuracy: 0.9276 - val_loss: 0.6174 - val_accuracy: 0.7871
Epoch 34/40
74/74 [==============================] - 15s 205ms/step - loss: 0.2184 - accuracy: 0.9268 - val_loss: 0.5795 - val_accuracy: 0.7922
Epoch 35/40
74/74 [==============================] - 16s 218ms/step - loss: 0.2221 - accuracy: 0.9242 - val_loss: 0.5509 - val_accuracy: 0.7836
Epoch 36/40
74/74 [==============================] - 15s 197ms/step - loss: 0.2219 - accuracy: 0.9268 - val_loss: 0.4793 - val_accuracy: 0.8177
Epoch 37/40
74/74 [==============================] - 15s 197ms/step - loss: 0.2011 - accuracy: 0.9383 - val_loss: 0.5662 - val_accuracy: 0.8075
Epoch 38/40
74/74 [==============================] - 15s 205ms/step - loss: 0.1995 - accuracy: 0.9361 - val_loss: 0.5317 - val_accuracy: 0.7973
Epoch 39/40
74/74 [==============================] - 16s 219ms/step - loss: 0.2013 - accuracy: 0.9353 - val_loss: 0.4922 - val_accuracy: 0.8126
Epoch 40/40
74/74 [==============================] - 15s 207ms/step - loss: 0.1946 - accuracy: 0.9378 - val_loss: 0.5878 - val_accuracy: 0.7990
In [62]:
# Visualization of the learning curves

plt.subplot(1, 2, 1)
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.ylim([0.0, 2.0])
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.legend(['train', 'valid'])

plt.subplot(1, 2, 2)
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.ylim([0.5, 1.0])
plt.xlabel('Epoch')
plt.ylabel('Accuracy')
plt.legend(['train', 'valid'])
Out[62]:
<matplotlib.legend.Legend at 0x29195eb15e0>

To fights this variability in validation accuracy as the epoches pass, we are going to introduce a time-decaying learning rate.

TimeDecay¶

In [15]:
# Architecture definition
inputs = keras.Input((None,None,3))

x = inputs

x = keras.layers.RandomCrop(224, 224)(x)
x = keras.layers.RandomFlip(mode='horizontal')(x)
x = keras.layers.RandomContrast(0.25)(x)

x = keras.layers.Conv2D(32, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.MaxPooling2D(3, strides=3, padding="same")(x)

x = keras.layers.Conv2D(64, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.MaxPooling2D(3, strides=3, padding="same")(x)

x = keras.layers.Conv2D(128, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)

x = keras.layers.GlobalMaxPooling2D()(x)

outputs = keras.layers.Dense(num_classes, activation="softmax")(x)
net = keras.Model(inputs, outputs)
In [16]:
callback = keras.callbacks.EarlyStopping(monitor='val_loss', patience=10)
In [17]:
lr_schedule = keras.optimizers.schedules.InverseTimeDecay(
    initial_learning_rate = 0.0005,
    decay_steps = 65,
    decay_rate = 0.08)
optimizer = keras.optimizers.RMSprop(learning_rate=lr_schedule)
net.compile(loss=keras.losses.categorical_crossentropy,
            optimizer=optimizer,
            metrics=['accuracy'])
In [18]:
# Model training
history = net.fit(train_ds_256,
          epochs=80,
          validation_data=val_ds,
            callbacks=[callback])

Epoch 1/80
65/65 [==============================] - 7s 91ms/step - loss: 1.2730 - accuracy: 0.5263 - val_loss: 1.6327 - val_accuracy: 0.3432
Epoch 2/80
65/65 [==============================] - 6s 87ms/step - loss: 0.8948 - accuracy: 0.6445 - val_loss: 1.0282 - val_accuracy: 0.5977
Epoch 3/80
65/65 [==============================] - 6s 87ms/step - loss: 0.7678 - accuracy: 0.7067 - val_loss: 0.7859 - val_accuracy: 0.7114
Epoch 4/80
65/65 [==============================] - 6s 92ms/step - loss: 0.7034 - accuracy: 0.7354 - val_loss: 0.8250 - val_accuracy: 0.6841
Epoch 5/80
65/65 [==============================] - 7s 100ms/step - loss: 0.6387 - accuracy: 0.7665 - val_loss: 0.7197 - val_accuracy: 0.7205
Epoch 6/80
65/65 [==============================] - 7s 103ms/step - loss: 0.6156 - accuracy: 0.7690 - val_loss: 0.8018 - val_accuracy: 0.7080
Epoch 7/80
65/65 [==============================] - 7s 105ms/step - loss: 0.5696 - accuracy: 0.7884 - val_loss: 0.8430 - val_accuracy: 0.6841
Epoch 8/80
65/65 [==============================] - 7s 104ms/step - loss: 0.5270 - accuracy: 0.8054 - val_loss: 0.6586 - val_accuracy: 0.7523
Epoch 9/80
65/65 [==============================] - 7s 104ms/step - loss: 0.4984 - accuracy: 0.8113 - val_loss: 0.6562 - val_accuracy: 0.7432
Epoch 10/80
65/65 [==============================] - 7s 105ms/step - loss: 0.5081 - accuracy: 0.8127 - val_loss: 0.7417 - val_accuracy: 0.7182
Epoch 11/80
65/65 [==============================] - 7s 107ms/step - loss: 0.4846 - accuracy: 0.8322 - val_loss: 0.7101 - val_accuracy: 0.7466
Epoch 12/80
65/65 [==============================] - 7s 108ms/step - loss: 0.4498 - accuracy: 0.8478 - val_loss: 0.5766 - val_accuracy: 0.7795
Epoch 13/80
65/65 [==============================] - 7s 109ms/step - loss: 0.4265 - accuracy: 0.8512 - val_loss: 0.6581 - val_accuracy: 0.7545
Epoch 14/80
65/65 [==============================] - 7s 105ms/step - loss: 0.4314 - accuracy: 0.8400 - val_loss: 0.5665 - val_accuracy: 0.7886
Epoch 15/80
65/65 [==============================] - 7s 106ms/step - loss: 0.4116 - accuracy: 0.8614 - val_loss: 0.5597 - val_accuracy: 0.7807
Epoch 16/80
65/65 [==============================] - 7s 106ms/step - loss: 0.3953 - accuracy: 0.8628 - val_loss: 0.5849 - val_accuracy: 0.7670
Epoch 17/80
65/65 [==============================] - 7s 106ms/step - loss: 0.3808 - accuracy: 0.8619 - val_loss: 0.5355 - val_accuracy: 0.7943
Epoch 18/80
65/65 [==============================] - 7s 109ms/step - loss: 0.3804 - accuracy: 0.8672 - val_loss: 0.5708 - val_accuracy: 0.7682
Epoch 19/80
65/65 [==============================] - 9s 132ms/step - loss: 0.3759 - accuracy: 0.8760 - val_loss: 0.6136 - val_accuracy: 0.7795
Epoch 20/80
65/65 [==============================] - 10s 156ms/step - loss: 0.3449 - accuracy: 0.8925 - val_loss: 0.5439 - val_accuracy: 0.7966
Epoch 21/80
65/65 [==============================] - 11s 174ms/step - loss: 0.3600 - accuracy: 0.8760 - val_loss: 0.5576 - val_accuracy: 0.7852
Epoch 22/80
65/65 [==============================] - 12s 181ms/step - loss: 0.3387 - accuracy: 0.8857 - val_loss: 0.6080 - val_accuracy: 0.7693
Epoch 23/80
65/65 [==============================] - 13s 202ms/step - loss: 0.3291 - accuracy: 0.8901 - val_loss: 0.5611 - val_accuracy: 0.7955
Epoch 24/80
65/65 [==============================] - 14s 207ms/step - loss: 0.3314 - accuracy: 0.8872 - val_loss: 0.5657 - val_accuracy: 0.7886
Epoch 25/80
65/65 [==============================] - 15s 229ms/step - loss: 0.3224 - accuracy: 0.8954 - val_loss: 0.5143 - val_accuracy: 0.8091
Epoch 26/80
65/65 [==============================] - 15s 223ms/step - loss: 0.3132 - accuracy: 0.8945 - val_loss: 0.5261 - val_accuracy: 0.8034
Epoch 27/80
65/65 [==============================] - 15s 232ms/step - loss: 0.3005 - accuracy: 0.9018 - val_loss: 0.5106 - val_accuracy: 0.8216
Epoch 28/80
65/65 [==============================] - 15s 235ms/step - loss: 0.3048 - accuracy: 0.9003 - val_loss: 0.5258 - val_accuracy: 0.8114
Epoch 29/80
65/65 [==============================] - 16s 238ms/step - loss: 0.2931 - accuracy: 0.9086 - val_loss: 0.5045 - val_accuracy: 0.8091
Epoch 30/80
65/65 [==============================] - 15s 236ms/step - loss: 0.2900 - accuracy: 0.9095 - val_loss: 0.5242 - val_accuracy: 0.7898
Epoch 31/80
65/65 [==============================] - 16s 237ms/step - loss: 0.2767 - accuracy: 0.9183 - val_loss: 0.5198 - val_accuracy: 0.8034
Epoch 32/80
65/65 [==============================] - 15s 236ms/step - loss: 0.2764 - accuracy: 0.9129 - val_loss: 0.4957 - val_accuracy: 0.8227
Epoch 33/80
65/65 [==============================] - 15s 236ms/step - loss: 0.2578 - accuracy: 0.9202 - val_loss: 0.5045 - val_accuracy: 0.8091
Epoch 34/80
65/65 [==============================] - 15s 236ms/step - loss: 0.2710 - accuracy: 0.9163 - val_loss: 0.5075 - val_accuracy: 0.8125
Epoch 35/80
65/65 [==============================] - 15s 236ms/step - loss: 0.2573 - accuracy: 0.9207 - val_loss: 0.4790 - val_accuracy: 0.8193
Epoch 36/80
65/65 [==============================] - 15s 237ms/step - loss: 0.2516 - accuracy: 0.9270 - val_loss: 0.6071 - val_accuracy: 0.7670
Epoch 37/80
65/65 [==============================] - 16s 237ms/step - loss: 0.2538 - accuracy: 0.9256 - val_loss: 0.5329 - val_accuracy: 0.8023
Epoch 38/80
65/65 [==============================] - 15s 237ms/step - loss: 0.2574 - accuracy: 0.9168 - val_loss: 0.5426 - val_accuracy: 0.7920
Epoch 39/80
65/65 [==============================] - 16s 237ms/step - loss: 0.2364 - accuracy: 0.9373 - val_loss: 0.5345 - val_accuracy: 0.7886
Epoch 40/80
65/65 [==============================] - 16s 237ms/step - loss: 0.2593 - accuracy: 0.9163 - val_loss: 0.5236 - val_accuracy: 0.8000
Epoch 41/80
65/65 [==============================] - 16s 237ms/step - loss: 0.2454 - accuracy: 0.9241 - val_loss: 0.4954 - val_accuracy: 0.8125
Epoch 42/80
65/65 [==============================] - 16s 237ms/step - loss: 0.2382 - accuracy: 0.9334 - val_loss: 0.5312 - val_accuracy: 0.8045
Epoch 43/80
65/65 [==============================] - 16s 237ms/step - loss: 0.2326 - accuracy: 0.9334 - val_loss: 0.5053 - val_accuracy: 0.8057
Epoch 44/80
65/65 [==============================] - 16s 237ms/step - loss: 0.2251 - accuracy: 0.9382 - val_loss: 0.4967 - val_accuracy: 0.8136
Epoch 45/80
65/65 [==============================] - 16s 238ms/step - loss: 0.2296 - accuracy: 0.9329 - val_loss: 0.4888 - val_accuracy: 0.8205
In [19]:
# Visualization of the learning curves

plt.subplot(1, 2, 1)
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.ylim([0.0, 2.0])
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.legend(['train', 'valid'])

plt.subplot(1, 2, 2)
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.ylim([0.5, 1.0])
plt.xlabel('Epoch')
plt.ylabel('Accuracy')
plt.legend(['train', 'valid'])
Out[19]:
<matplotlib.legend.Legend at 0x216063bdfa0>

The addition of the Time-decaying Learning Rate helped to increase the accuracy of the validation set even at higher epochs.

Adam optimizer¶

In [30]:
# Architecture definition
inputs = keras.Input((None,None,3))

x = inputs

x = keras.layers.RandomCrop(224, 224)(x)
x = keras.layers.RandomFlip(mode='horizontal')(x)
x = keras.layers.RandomContrast(0.25)(x)

x = keras.layers.Conv2D(32, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.MaxPooling2D(3, strides=3, padding="same")(x)

x = keras.layers.Conv2D(64, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.MaxPooling2D(3, strides=3, padding="same")(x)

x = keras.layers.Conv2D(128, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)

x = keras.layers.GlobalMaxPooling2D()(x)

outputs = keras.layers.Dense(num_classes, activation="softmax")(x)
net = keras.Model(inputs, outputs)
In [31]:
callback = keras.callbacks.EarlyStopping(monitor='val_loss', patience=10)
In [32]:
lr_schedule = keras.optimizers.schedules.InverseTimeDecay(
    initial_learning_rate = 0.0005,
    decay_steps = 65,
    decay_rate = 0.08)
optimizer = keras.optimizers.Adam(learning_rate=lr_schedule)
net.compile(loss=keras.losses.categorical_crossentropy,
            optimizer=optimizer,
            metrics=['accuracy'])
In [33]:
# Model training
history = net.fit(train_ds_256,
          epochs=80,
          validation_data=val_ds,
            callbacks=[callback])

Epoch 1/80
65/65 [==============================] - 7s 86ms/step - loss: 1.4620 - accuracy: 0.4912 - val_loss: 1.4347 - val_accuracy: 0.4364
Epoch 2/80
65/65 [==============================] - 6s 85ms/step - loss: 0.8596 - accuracy: 0.6644 - val_loss: 0.9720 - val_accuracy: 0.6023
Epoch 3/80
65/65 [==============================] - 6s 84ms/step - loss: 0.7387 - accuracy: 0.7237 - val_loss: 0.7942 - val_accuracy: 0.7114
Epoch 4/80
65/65 [==============================] - 6s 85ms/step - loss: 0.6981 - accuracy: 0.7349 - val_loss: 0.7216 - val_accuracy: 0.7420
Epoch 5/80
65/65 [==============================] - 6s 85ms/step - loss: 0.6538 - accuracy: 0.7554 - val_loss: 0.7498 - val_accuracy: 0.7182
Epoch 6/80
65/65 [==============================] - 6s 87ms/step - loss: 0.5931 - accuracy: 0.7899 - val_loss: 0.7111 - val_accuracy: 0.7295
Epoch 7/80
65/65 [==============================] - 6s 86ms/step - loss: 0.5946 - accuracy: 0.7865 - val_loss: 0.7053 - val_accuracy: 0.7420
Epoch 8/80
65/65 [==============================] - 6s 86ms/step - loss: 0.5534 - accuracy: 0.8025 - val_loss: 0.7194 - val_accuracy: 0.7170
Epoch 9/80
65/65 [==============================] - 6s 86ms/step - loss: 0.5324 - accuracy: 0.8084 - val_loss: 0.7671 - val_accuracy: 0.7182
Epoch 10/80
65/65 [==============================] - 6s 88ms/step - loss: 0.4909 - accuracy: 0.8268 - val_loss: 0.6170 - val_accuracy: 0.7682
Epoch 11/80
65/65 [==============================] - 6s 92ms/step - loss: 0.4690 - accuracy: 0.8419 - val_loss: 0.6160 - val_accuracy: 0.7705
Epoch 12/80
65/65 [==============================] - 6s 95ms/step - loss: 0.4457 - accuracy: 0.8487 - val_loss: 0.5988 - val_accuracy: 0.7830
Epoch 13/80
65/65 [==============================] - 7s 102ms/step - loss: 0.4416 - accuracy: 0.8434 - val_loss: 0.6491 - val_accuracy: 0.7614
Epoch 14/80
65/65 [==============================] - 7s 103ms/step - loss: 0.4405 - accuracy: 0.8487 - val_loss: 0.5738 - val_accuracy: 0.7841
Epoch 15/80
65/65 [==============================] - 7s 104ms/step - loss: 0.4155 - accuracy: 0.8580 - val_loss: 0.5651 - val_accuracy: 0.7966
Epoch 16/80
65/65 [==============================] - 7s 104ms/step - loss: 0.4064 - accuracy: 0.8624 - val_loss: 0.6185 - val_accuracy: 0.7625
Epoch 17/80
65/65 [==============================] - 7s 104ms/step - loss: 0.4041 - accuracy: 0.8604 - val_loss: 0.5421 - val_accuracy: 0.7852
Epoch 18/80
65/65 [==============================] - 7s 104ms/step - loss: 0.3901 - accuracy: 0.8658 - val_loss: 0.5831 - val_accuracy: 0.7898
Epoch 19/80
65/65 [==============================] - 7s 104ms/step - loss: 0.3833 - accuracy: 0.8716 - val_loss: 0.5546 - val_accuracy: 0.7932
Epoch 20/80
65/65 [==============================] - 7s 104ms/step - loss: 0.3771 - accuracy: 0.8726 - val_loss: 0.5794 - val_accuracy: 0.7898
Epoch 21/80
65/65 [==============================] - 7s 104ms/step - loss: 0.3504 - accuracy: 0.8842 - val_loss: 0.5652 - val_accuracy: 0.7795
Epoch 22/80
65/65 [==============================] - 7s 104ms/step - loss: 0.3554 - accuracy: 0.8784 - val_loss: 0.5040 - val_accuracy: 0.8102
Epoch 23/80
65/65 [==============================] - 7s 104ms/step - loss: 0.3383 - accuracy: 0.8911 - val_loss: 0.5171 - val_accuracy: 0.8045
Epoch 24/80
65/65 [==============================] - 7s 104ms/step - loss: 0.3350 - accuracy: 0.8881 - val_loss: 0.5052 - val_accuracy: 0.8102
Epoch 25/80
65/65 [==============================] - 7s 104ms/step - loss: 0.3390 - accuracy: 0.8935 - val_loss: 0.4899 - val_accuracy: 0.8170
Epoch 26/80
65/65 [==============================] - 7s 104ms/step - loss: 0.3239 - accuracy: 0.8998 - val_loss: 0.5045 - val_accuracy: 0.8057
Epoch 27/80
65/65 [==============================] - 7s 104ms/step - loss: 0.3203 - accuracy: 0.8964 - val_loss: 0.4735 - val_accuracy: 0.8261
Epoch 28/80
65/65 [==============================] - 7s 105ms/step - loss: 0.3170 - accuracy: 0.9003 - val_loss: 0.5107 - val_accuracy: 0.8205
Epoch 29/80
65/65 [==============================] - 7s 107ms/step - loss: 0.3026 - accuracy: 0.9018 - val_loss: 0.5015 - val_accuracy: 0.8273
Epoch 30/80
65/65 [==============================] - 7s 105ms/step - loss: 0.2982 - accuracy: 0.9047 - val_loss: 0.4851 - val_accuracy: 0.8102
Epoch 31/80
65/65 [==============================] - 7s 104ms/step - loss: 0.2923 - accuracy: 0.9134 - val_loss: 0.5089 - val_accuracy: 0.8080
Epoch 32/80
65/65 [==============================] - 7s 105ms/step - loss: 0.2791 - accuracy: 0.9110 - val_loss: 0.5080 - val_accuracy: 0.8205
Epoch 33/80
65/65 [==============================] - 7s 105ms/step - loss: 0.2858 - accuracy: 0.9042 - val_loss: 0.4798 - val_accuracy: 0.8250
Epoch 34/80
65/65 [==============================] - 7s 104ms/step - loss: 0.2953 - accuracy: 0.9061 - val_loss: 0.5145 - val_accuracy: 0.8034
Epoch 35/80
65/65 [==============================] - 7s 105ms/step - loss: 0.2820 - accuracy: 0.9076 - val_loss: 0.4725 - val_accuracy: 0.8193
Epoch 36/80
65/65 [==============================] - 7s 105ms/step - loss: 0.2889 - accuracy: 0.9071 - val_loss: 0.4865 - val_accuracy: 0.8193
Epoch 37/80
65/65 [==============================] - 7s 105ms/step - loss: 0.2847 - accuracy: 0.9095 - val_loss: 0.4739 - val_accuracy: 0.8227
Epoch 38/80
65/65 [==============================] - 7s 105ms/step - loss: 0.2686 - accuracy: 0.9227 - val_loss: 0.4883 - val_accuracy: 0.8227
Epoch 39/80
65/65 [==============================] - 7s 105ms/step - loss: 0.2637 - accuracy: 0.9212 - val_loss: 0.4806 - val_accuracy: 0.8239
Epoch 40/80
65/65 [==============================] - 7s 105ms/step - loss: 0.2555 - accuracy: 0.9280 - val_loss: 0.4735 - val_accuracy: 0.8318
Epoch 41/80
65/65 [==============================] - 7s 106ms/step - loss: 0.2522 - accuracy: 0.9236 - val_loss: 0.4649 - val_accuracy: 0.8318
Epoch 42/80
65/65 [==============================] - 7s 106ms/step - loss: 0.2510 - accuracy: 0.9285 - val_loss: 0.4739 - val_accuracy: 0.8205
Epoch 43/80
65/65 [==============================] - 7s 105ms/step - loss: 0.2530 - accuracy: 0.9261 - val_loss: 0.4700 - val_accuracy: 0.8239
Epoch 44/80
65/65 [==============================] - 7s 105ms/step - loss: 0.2446 - accuracy: 0.9222 - val_loss: 0.4673 - val_accuracy: 0.8227
Epoch 45/80
65/65 [==============================] - 7s 104ms/step - loss: 0.2506 - accuracy: 0.9251 - val_loss: 0.4629 - val_accuracy: 0.8295
Epoch 46/80
65/65 [==============================] - 7s 105ms/step - loss: 0.2394 - accuracy: 0.9246 - val_loss: 0.4609 - val_accuracy: 0.8352
Epoch 47/80
65/65 [==============================] - 7s 106ms/step - loss: 0.2323 - accuracy: 0.9397 - val_loss: 0.4955 - val_accuracy: 0.8205
Epoch 48/80
65/65 [==============================] - 7s 111ms/step - loss: 0.2475 - accuracy: 0.9241 - val_loss: 0.4819 - val_accuracy: 0.8170
Epoch 49/80
65/65 [==============================] - 8s 116ms/step - loss: 0.2286 - accuracy: 0.9329 - val_loss: 0.4684 - val_accuracy: 0.8295
Epoch 50/80
65/65 [==============================] - 8s 119ms/step - loss: 0.2250 - accuracy: 0.9392 - val_loss: 0.4593 - val_accuracy: 0.8307
Epoch 51/80
65/65 [==============================] - 8s 121ms/step - loss: 0.2306 - accuracy: 0.9295 - val_loss: 0.4683 - val_accuracy: 0.8261
Epoch 52/80
65/65 [==============================] - 8s 125ms/step - loss: 0.2263 - accuracy: 0.9387 - val_loss: 0.4503 - val_accuracy: 0.8375
Epoch 53/80
65/65 [==============================] - 8s 126ms/step - loss: 0.2178 - accuracy: 0.9470 - val_loss: 0.4706 - val_accuracy: 0.8273
Epoch 54/80
65/65 [==============================] - 9s 140ms/step - loss: 0.2223 - accuracy: 0.9402 - val_loss: 0.4532 - val_accuracy: 0.8352
Epoch 55/80
65/65 [==============================] - 8s 128ms/step - loss: 0.2165 - accuracy: 0.9416 - val_loss: 0.4444 - val_accuracy: 0.8375
Epoch 56/80
65/65 [==============================] - 9s 145ms/step - loss: 0.2212 - accuracy: 0.9314 - val_loss: 0.4455 - val_accuracy: 0.8420
Epoch 57/80
65/65 [==============================] - 9s 144ms/step - loss: 0.2169 - accuracy: 0.9441 - val_loss: 0.4498 - val_accuracy: 0.8341
Epoch 58/80
65/65 [==============================] - 9s 145ms/step - loss: 0.2058 - accuracy: 0.9426 - val_loss: 0.4391 - val_accuracy: 0.8409
Epoch 59/80
65/65 [==============================] - 9s 138ms/step - loss: 0.2171 - accuracy: 0.9377 - val_loss: 0.4656 - val_accuracy: 0.8250
Epoch 60/80
65/65 [==============================] - 9s 144ms/step - loss: 0.2022 - accuracy: 0.9553 - val_loss: 0.4503 - val_accuracy: 0.8386
Epoch 61/80
65/65 [==============================] - 10s 148ms/step - loss: 0.2149 - accuracy: 0.9407 - val_loss: 0.4642 - val_accuracy: 0.8216
Epoch 62/80
65/65 [==============================] - 10s 146ms/step - loss: 0.1976 - accuracy: 0.9416 - val_loss: 0.4440 - val_accuracy: 0.8330
Epoch 63/80
65/65 [==============================] - 10s 150ms/step - loss: 0.2050 - accuracy: 0.9421 - val_loss: 0.4419 - val_accuracy: 0.8364
Epoch 64/80
65/65 [==============================] - 10s 148ms/step - loss: 0.1968 - accuracy: 0.9465 - val_loss: 0.4427 - val_accuracy: 0.8341
Epoch 65/80
65/65 [==============================] - 10s 153ms/step - loss: 0.1978 - accuracy: 0.9446 - val_loss: 0.4402 - val_accuracy: 0.8455
Epoch 66/80
65/65 [==============================] - 10s 152ms/step - loss: 0.1881 - accuracy: 0.9523 - val_loss: 0.4467 - val_accuracy: 0.8375
Epoch 67/80
65/65 [==============================] - 10s 159ms/step - loss: 0.1904 - accuracy: 0.9528 - val_loss: 0.4531 - val_accuracy: 0.8250
Epoch 68/80
65/65 [==============================] - 11s 164ms/step - loss: 0.2093 - accuracy: 0.9373 - val_loss: 0.4409 - val_accuracy: 0.8341
In [34]:
# Visualization of the learning curves

plt.subplot(1, 2, 1)
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.ylim([0.0, 2.0])
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.legend(['train', 'valid'])

plt.subplot(1, 2, 2)
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.ylim([0.5, 1.0])
plt.xlabel('Epoch')
plt.ylabel('Accuracy')
plt.legend(['train', 'valid'])
Out[34]:
<matplotlib.legend.Legend at 0x2301fae4670>

By changing the optimizer to Adam, we have a notable increase in validation accuracy.
Even though the model is not technically overfitting, since the two curves are not diverging, we are going to test some Regularization methods. We do this in the hope that, by closing the gap between validation and train, we will be able to increase the validation accuracy further.

Try Regularizer¶

0.001¶

In [ ]:
# Architecture definition
inputs = keras.Input((None,None,3))

x = inputs

x = keras.layers.RandomCrop(224, 224)(x)
x = keras.layers.RandomFlip(mode='horizontal')(x)
x = keras.layers.RandomContrast(0.25)(x)

x = keras.layers.Conv2D(32, 3, padding="same", kernel_regularizer=tf.keras.regularizers.l2(0.001))(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.MaxPooling2D(3, strides=3, padding="same")(x)

x = keras.layers.Conv2D(64, 3, padding="same", kernel_regularizer=tf.keras.regularizers.l2(0.001))(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.MaxPooling2D(3, strides=3, padding="same")(x)

x = keras.layers.Conv2D(128, 3, padding="same", kernel_regularizer=tf.keras.regularizers.l2(0.001))(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)

x = keras.layers.GlobalMaxPooling2D()(x)

outputs = keras.layers.Dense(num_classes, activation="softmax", kernel_regularizer=tf.keras.regularizers.l2(0.001))(x)
net = keras.Model(inputs, outputs)
In [ ]:
callback = keras.callbacks.EarlyStopping(monitor='val_loss', patience=10)
In [ ]:
lr_schedule = keras.optimizers.schedules.InverseTimeDecay(
    initial_learning_rate = 0.0005,
    decay_steps = 65,
    decay_rate = 0.08)
optimizer = keras.optimizers.Adam(learning_rate=lr_schedule)
net.compile(loss=keras.losses.categorical_crossentropy,
            optimizer=optimizer,
            metrics=['accuracy'])
In [ ]:
# Model training
history = net.fit(train_ds_256,
          epochs=80,
          validation_data=val_ds,
            callbacks=[callback])

Epoch 1/80
65/65 [==============================] - 7s 90ms/step - loss: 1.4885 - accuracy: 0.5019 - val_loss: 3.1283 - val_accuracy: 0.2250
Epoch 2/80
65/65 [==============================] - 6s 86ms/step - loss: 1.0031 - accuracy: 0.6615 - val_loss: 1.7334 - val_accuracy: 0.3716
Epoch 3/80
65/65 [==============================] - 6s 86ms/step - loss: 0.9414 - accuracy: 0.6980 - val_loss: 1.0869 - val_accuracy: 0.6523
Epoch 4/80
65/65 [==============================] - 6s 86ms/step - loss: 0.8583 - accuracy: 0.7247 - val_loss: 0.8709 - val_accuracy: 0.7330
Epoch 5/80
65/65 [==============================] - 6s 88ms/step - loss: 0.8036 - accuracy: 0.7466 - val_loss: 0.9929 - val_accuracy: 0.6648
Epoch 6/80
65/65 [==============================] - 6s 98ms/step - loss: 0.7671 - accuracy: 0.7665 - val_loss: 0.8278 - val_accuracy: 0.7364
Epoch 7/80
65/65 [==============================] - 7s 105ms/step - loss: 0.7624 - accuracy: 0.7690 - val_loss: 0.8384 - val_accuracy: 0.7250
Epoch 8/80
65/65 [==============================] - 7s 106ms/step - loss: 0.7317 - accuracy: 0.7806 - val_loss: 0.8585 - val_accuracy: 0.7205
Epoch 9/80
65/65 [==============================] - 7s 106ms/step - loss: 0.7002 - accuracy: 0.7913 - val_loss: 0.9113 - val_accuracy: 0.7091
Epoch 10/80
65/65 [==============================] - 7s 106ms/step - loss: 0.6707 - accuracy: 0.8084 - val_loss: 0.7858 - val_accuracy: 0.7557
Epoch 11/80
65/65 [==============================] - 7s 105ms/step - loss: 0.6332 - accuracy: 0.8171 - val_loss: 0.7403 - val_accuracy: 0.7795
Epoch 12/80
65/65 [==============================] - 7s 106ms/step - loss: 0.6100 - accuracy: 0.8361 - val_loss: 0.7530 - val_accuracy: 0.7568
Epoch 13/80
65/65 [==============================] - 7s 106ms/step - loss: 0.6147 - accuracy: 0.8303 - val_loss: 0.7855 - val_accuracy: 0.7602
Epoch 14/80
65/65 [==============================] - 7s 106ms/step - loss: 0.5925 - accuracy: 0.8448 - val_loss: 0.7242 - val_accuracy: 0.7784
Epoch 15/80
65/65 [==============================] - 7s 106ms/step - loss: 0.5762 - accuracy: 0.8424 - val_loss: 0.7965 - val_accuracy: 0.7489
Epoch 16/80
65/65 [==============================] - 7s 106ms/step - loss: 0.5733 - accuracy: 0.8492 - val_loss: 0.7465 - val_accuracy: 0.7705
Epoch 17/80
65/65 [==============================] - 7s 109ms/step - loss: 0.5488 - accuracy: 0.8662 - val_loss: 0.8506 - val_accuracy: 0.7477
Epoch 18/80
65/65 [==============================] - 7s 106ms/step - loss: 0.5307 - accuracy: 0.8614 - val_loss: 0.7129 - val_accuracy: 0.7943
Epoch 19/80
65/65 [==============================] - 7s 108ms/step - loss: 0.5437 - accuracy: 0.8541 - val_loss: 0.7134 - val_accuracy: 0.7886
Epoch 20/80
65/65 [==============================] - 7s 106ms/step - loss: 0.5233 - accuracy: 0.8677 - val_loss: 0.6684 - val_accuracy: 0.7886
Epoch 21/80
65/65 [==============================] - 7s 107ms/step - loss: 0.5084 - accuracy: 0.8638 - val_loss: 0.6343 - val_accuracy: 0.8125
Epoch 22/80
65/65 [==============================] - 7s 106ms/step - loss: 0.4909 - accuracy: 0.8847 - val_loss: 0.6889 - val_accuracy: 0.7955
Epoch 23/80
65/65 [==============================] - 7s 106ms/step - loss: 0.4905 - accuracy: 0.8794 - val_loss: 0.6515 - val_accuracy: 0.8011
Epoch 24/80
65/65 [==============================] - 7s 106ms/step - loss: 0.4798 - accuracy: 0.8857 - val_loss: 0.6662 - val_accuracy: 0.8170
Epoch 25/80
65/65 [==============================] - 8s 115ms/step - loss: 0.4796 - accuracy: 0.8867 - val_loss: 0.6641 - val_accuracy: 0.7943
Epoch 26/80
65/65 [==============================] - 8s 125ms/step - loss: 0.4710 - accuracy: 0.8920 - val_loss: 0.6397 - val_accuracy: 0.8125
Epoch 27/80
65/65 [==============================] - 8s 128ms/step - loss: 0.4678 - accuracy: 0.8911 - val_loss: 0.6581 - val_accuracy: 0.8091
Epoch 28/80
65/65 [==============================] - 9s 137ms/step - loss: 0.4585 - accuracy: 0.8959 - val_loss: 0.6495 - val_accuracy: 0.8182
Epoch 29/80
65/65 [==============================] - 9s 135ms/step - loss: 0.4470 - accuracy: 0.9003 - val_loss: 0.6299 - val_accuracy: 0.8136
Epoch 30/80
65/65 [==============================] - 9s 145ms/step - loss: 0.4432 - accuracy: 0.9066 - val_loss: 0.6321 - val_accuracy: 0.8102
Epoch 31/80
65/65 [==============================] - 10s 147ms/step - loss: 0.4442 - accuracy: 0.9022 - val_loss: 0.6523 - val_accuracy: 0.8023
Epoch 32/80
65/65 [==============================] - 10s 155ms/step - loss: 0.4333 - accuracy: 0.9037 - val_loss: 0.6592 - val_accuracy: 0.7943
Epoch 33/80
65/65 [==============================] - 11s 173ms/step - loss: 0.4270 - accuracy: 0.9129 - val_loss: 0.6083 - val_accuracy: 0.8205
Epoch 34/80
65/65 [==============================] - 11s 173ms/step - loss: 0.4238 - accuracy: 0.9090 - val_loss: 0.6053 - val_accuracy: 0.8193
Epoch 35/80
65/65 [==============================] - 12s 185ms/step - loss: 0.4117 - accuracy: 0.9149 - val_loss: 0.6022 - val_accuracy: 0.8216
Epoch 36/80
65/65 [==============================] - 11s 175ms/step - loss: 0.4157 - accuracy: 0.9115 - val_loss: 0.6388 - val_accuracy: 0.8216
Epoch 37/80
65/65 [==============================] - 13s 193ms/step - loss: 0.4042 - accuracy: 0.9173 - val_loss: 0.6435 - val_accuracy: 0.8182
Epoch 38/80
65/65 [==============================] - 13s 198ms/step - loss: 0.3951 - accuracy: 0.9261 - val_loss: 0.6061 - val_accuracy: 0.8091
Epoch 39/80
65/65 [==============================] - 12s 185ms/step - loss: 0.4038 - accuracy: 0.9149 - val_loss: 0.5985 - val_accuracy: 0.8239
Epoch 40/80
65/65 [==============================] - 12s 188ms/step - loss: 0.3848 - accuracy: 0.9232 - val_loss: 0.5964 - val_accuracy: 0.8284
Epoch 41/80
65/65 [==============================] - 14s 209ms/step - loss: 0.3813 - accuracy: 0.9314 - val_loss: 0.6402 - val_accuracy: 0.8193
Epoch 42/80
65/65 [==============================] - 14s 214ms/step - loss: 0.3785 - accuracy: 0.9304 - val_loss: 0.6061 - val_accuracy: 0.8170
Epoch 43/80
65/65 [==============================] - 13s 200ms/step - loss: 0.3761 - accuracy: 0.9334 - val_loss: 0.5777 - val_accuracy: 0.8341
Epoch 44/80
65/65 [==============================] - 14s 215ms/step - loss: 0.3786 - accuracy: 0.9232 - val_loss: 0.5844 - val_accuracy: 0.8307
Epoch 45/80
65/65 [==============================] - 14s 216ms/step - loss: 0.3671 - accuracy: 0.9290 - val_loss: 0.5845 - val_accuracy: 0.8307
Epoch 46/80
65/65 [==============================] - 13s 203ms/step - loss: 0.3707 - accuracy: 0.9368 - val_loss: 0.5745 - val_accuracy: 0.8409
Epoch 47/80
65/65 [==============================] - 14s 217ms/step - loss: 0.3627 - accuracy: 0.9353 - val_loss: 0.5805 - val_accuracy: 0.8364
Epoch 48/80
65/65 [==============================] - 15s 223ms/step - loss: 0.3617 - accuracy: 0.9339 - val_loss: 0.5805 - val_accuracy: 0.8364
Epoch 49/80
65/65 [==============================] - 15s 228ms/step - loss: 0.3571 - accuracy: 0.9392 - val_loss: 0.5781 - val_accuracy: 0.8284
Epoch 50/80
65/65 [==============================] - 15s 233ms/step - loss: 0.3675 - accuracy: 0.9358 - val_loss: 0.5962 - val_accuracy: 0.8170
Epoch 51/80
65/65 [==============================] - 15s 235ms/step - loss: 0.3540 - accuracy: 0.9446 - val_loss: 0.5830 - val_accuracy: 0.8250
Epoch 52/80
65/65 [==============================] - 16s 238ms/step - loss: 0.3491 - accuracy: 0.9446 - val_loss: 0.5673 - val_accuracy: 0.8341
Epoch 53/80
65/65 [==============================] - 15s 237ms/step - loss: 0.3478 - accuracy: 0.9411 - val_loss: 0.5667 - val_accuracy: 0.8341
Epoch 54/80
65/65 [==============================] - 15s 235ms/step - loss: 0.3479 - accuracy: 0.9397 - val_loss: 0.5665 - val_accuracy: 0.8409
Epoch 55/80
65/65 [==============================] - 15s 237ms/step - loss: 0.3496 - accuracy: 0.9363 - val_loss: 0.5781 - val_accuracy: 0.8398
Epoch 56/80
65/65 [==============================] - 15s 237ms/step - loss: 0.3441 - accuracy: 0.9407 - val_loss: 0.5743 - val_accuracy: 0.8364
Epoch 57/80
65/65 [==============================] - 15s 237ms/step - loss: 0.3459 - accuracy: 0.9358 - val_loss: 0.5538 - val_accuracy: 0.8432
Epoch 58/80
65/65 [==============================] - 15s 237ms/step - loss: 0.3439 - accuracy: 0.9402 - val_loss: 0.5568 - val_accuracy: 0.8318
Epoch 59/80
65/65 [==============================] - 15s 237ms/step - loss: 0.3280 - accuracy: 0.9538 - val_loss: 0.5593 - val_accuracy: 0.8307
Epoch 60/80
65/65 [==============================] - 15s 237ms/step - loss: 0.3254 - accuracy: 0.9470 - val_loss: 0.5715 - val_accuracy: 0.8284
Epoch 61/80
65/65 [==============================] - 15s 237ms/step - loss: 0.3357 - accuracy: 0.9436 - val_loss: 0.6162 - val_accuracy: 0.8080
Epoch 62/80
65/65 [==============================] - 15s 237ms/step - loss: 0.3232 - accuracy: 0.9514 - val_loss: 0.5827 - val_accuracy: 0.8295
Epoch 63/80
65/65 [==============================] - 15s 237ms/step - loss: 0.3249 - accuracy: 0.9475 - val_loss: 0.5676 - val_accuracy: 0.8318
Epoch 64/80
65/65 [==============================] - 15s 237ms/step - loss: 0.3285 - accuracy: 0.9504 - val_loss: 0.5768 - val_accuracy: 0.8307
Epoch 65/80
65/65 [==============================] - 15s 237ms/step - loss: 0.3210 - accuracy: 0.9533 - val_loss: 0.5639 - val_accuracy: 0.8261
Epoch 66/80
65/65 [==============================] - 15s 237ms/step - loss: 0.3192 - accuracy: 0.9504 - val_loss: 0.5715 - val_accuracy: 0.8261
Epoch 67/80
65/65 [==============================] - 15s 237ms/step - loss: 0.3232 - accuracy: 0.9504 - val_loss: 0.5547 - val_accuracy: 0.8455
In [ ]:
# Visualization of the learning curves

plt.subplot(1, 2, 1)
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.ylim([0.0, 2.0])
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.legend(['train', 'valid'])

plt.subplot(1, 2, 2)
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.ylim([0.5, 1.0])
plt.xlabel('Epoch')
plt.ylabel('Accuracy')
plt.legend(['train', 'valid'])

<matplotlib.legend.Legend at 0x230205357f0>

The addition of a regularizer does not seem to benefit the model in a notable way. Other attempts were made (changing the parameter, using the L1 regularizer) but none of them brought a substantial improvement in the model quality. We are then going to try another regularization method: DropOut layers.

DropOut¶

After Testing different combinations of Dropout layers, two were selected:

  1. 0.1 Dropout on second layer + 0.2 Dropout on third layer
  2. 0.1 Dropout on first layer + 0.2 Dropout on second layer + 0.3 Dropout on third layer

First combination (0.0 + 0.1 + 0.2)¶

In [ ]:
# Architecture definition
inputs = keras.Input((None,None,3))

x = inputs

x = keras.layers.RandomCrop(224, 224)(x)
x = keras.layers.RandomFlip(mode='horizontal')(x)
x = keras.layers.RandomContrast(0.25)(x)

x = keras.layers.Conv2D(32, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.MaxPooling2D(3, strides=3, padding="same")(x)

x = keras.layers.Conv2D(64, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.MaxPooling2D(3, strides=3, padding="same")(x)
x = keras.layers.Dropout(0.1)(x)

x = keras.layers.Conv2D(128, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.Dropout(0.2)(x)

x = keras.layers.GlobalMaxPooling2D()(x)

outputs = keras.layers.Dense(num_classes, activation="softmax")(x)
net = keras.Model(inputs, outputs)
In [ ]:
callback = keras.callbacks.EarlyStopping(monitor='val_loss', patience=10)
In [ ]:
lr_schedule = keras.optimizers.schedules.InverseTimeDecay(
    initial_learning_rate = 0.0005,
    decay_steps = 65,
    decay_rate = 0.08)
optimizer = keras.optimizers.Adam(learning_rate=lr_schedule)
net.compile(loss=keras.losses.categorical_crossentropy,
            optimizer=optimizer,
            metrics=['accuracy'])
In [ ]:
# Model training
history = net.fit(train_ds_256,
          epochs=80,
          validation_data=val_ds,
            callbacks=[callback])

Epoch 1/80
65/65 [==============================] - 14s 194ms/step - loss: 1.4038 - accuracy: 0.5049 - val_loss: 1.5637 - val_accuracy: 0.4295
Epoch 2/80
65/65 [==============================] - 14s 215ms/step - loss: 0.9336 - accuracy: 0.6411 - val_loss: 1.1013 - val_accuracy: 0.5659
Epoch 3/80
65/65 [==============================] - 13s 197ms/step - loss: 0.8578 - accuracy: 0.6697 - val_loss: 0.9209 - val_accuracy: 0.6307
Epoch 4/80
65/65 [==============================] - 14s 213ms/step - loss: 0.7904 - accuracy: 0.7004 - val_loss: 0.7873 - val_accuracy: 0.7057
Epoch 5/80
65/65 [==============================] - 14s 213ms/step - loss: 0.7285 - accuracy: 0.7257 - val_loss: 0.7394 - val_accuracy: 0.7261
Epoch 6/80
65/65 [==============================] - 15s 225ms/step - loss: 0.7166 - accuracy: 0.7354 - val_loss: 0.7698 - val_accuracy: 0.7170
Epoch 7/80
65/65 [==============================] - 15s 229ms/step - loss: 0.6774 - accuracy: 0.7485 - val_loss: 0.7441 - val_accuracy: 0.7148
Epoch 8/80
65/65 [==============================] - 15s 234ms/step - loss: 0.6341 - accuracy: 0.7714 - val_loss: 0.6918 - val_accuracy: 0.7398
Epoch 9/80
65/65 [==============================] - 14s 215ms/step - loss: 0.5985 - accuracy: 0.7797 - val_loss: 0.6602 - val_accuracy: 0.7557
Epoch 10/80
65/65 [==============================] - 15s 227ms/step - loss: 0.6166 - accuracy: 0.7695 - val_loss: 0.6884 - val_accuracy: 0.7591
Epoch 11/80
65/65 [==============================] - 15s 223ms/step - loss: 0.5901 - accuracy: 0.7821 - val_loss: 0.6280 - val_accuracy: 0.7716
Epoch 12/80
65/65 [==============================] - 15s 223ms/step - loss: 0.5737 - accuracy: 0.8001 - val_loss: 0.6202 - val_accuracy: 0.7750
Epoch 13/80
65/65 [==============================] - 15s 235ms/step - loss: 0.5678 - accuracy: 0.7879 - val_loss: 0.6162 - val_accuracy: 0.7761
Epoch 14/80
65/65 [==============================] - 15s 228ms/step - loss: 0.5573 - accuracy: 0.8001 - val_loss: 0.6524 - val_accuracy: 0.7568
Epoch 15/80
65/65 [==============================] - 14s 218ms/step - loss: 0.5237 - accuracy: 0.8113 - val_loss: 0.5988 - val_accuracy: 0.8000
Epoch 16/80
65/65 [==============================] - 15s 238ms/step - loss: 0.5083 - accuracy: 0.8191 - val_loss: 0.6159 - val_accuracy: 0.7727
Epoch 17/80
65/65 [==============================] - 15s 235ms/step - loss: 0.4977 - accuracy: 0.8181 - val_loss: 0.5977 - val_accuracy: 0.7818
Epoch 18/80
65/65 [==============================] - 15s 235ms/step - loss: 0.4974 - accuracy: 0.8186 - val_loss: 0.5990 - val_accuracy: 0.7875
Epoch 19/80
65/65 [==============================] - 15s 225ms/step - loss: 0.4674 - accuracy: 0.8337 - val_loss: 0.6037 - val_accuracy: 0.7693
Epoch 20/80
65/65 [==============================] - 15s 229ms/step - loss: 0.4758 - accuracy: 0.8239 - val_loss: 0.5870 - val_accuracy: 0.7886
Epoch 21/80
65/65 [==============================] - 16s 243ms/step - loss: 0.4651 - accuracy: 0.8322 - val_loss: 0.5723 - val_accuracy: 0.7875
Epoch 22/80
65/65 [==============================] - 15s 238ms/step - loss: 0.4801 - accuracy: 0.8259 - val_loss: 0.6195 - val_accuracy: 0.7659
Epoch 23/80
65/65 [==============================] - 15s 228ms/step - loss: 0.4536 - accuracy: 0.8322 - val_loss: 0.5850 - val_accuracy: 0.7841
Epoch 24/80
65/65 [==============================] - 16s 240ms/step - loss: 0.4381 - accuracy: 0.8473 - val_loss: 0.5656 - val_accuracy: 0.7852
Epoch 25/80
65/65 [==============================] - 15s 230ms/step - loss: 0.4278 - accuracy: 0.8444 - val_loss: 0.5640 - val_accuracy: 0.7795
Epoch 26/80
65/65 [==============================] - 15s 238ms/step - loss: 0.4253 - accuracy: 0.8551 - val_loss: 0.5465 - val_accuracy: 0.8023
Epoch 27/80
65/65 [==============================] - 15s 236ms/step - loss: 0.4276 - accuracy: 0.8502 - val_loss: 0.5438 - val_accuracy: 0.8170
Epoch 28/80
65/65 [==============================] - 16s 238ms/step - loss: 0.4223 - accuracy: 0.8531 - val_loss: 0.5372 - val_accuracy: 0.8034
Epoch 29/80
65/65 [==============================] - 15s 236ms/step - loss: 0.4192 - accuracy: 0.8482 - val_loss: 0.5344 - val_accuracy: 0.8102
Epoch 30/80
65/65 [==============================] - 15s 238ms/step - loss: 0.4092 - accuracy: 0.8492 - val_loss: 0.5235 - val_accuracy: 0.8102
Epoch 31/80
65/65 [==============================] - 15s 236ms/step - loss: 0.3964 - accuracy: 0.8604 - val_loss: 0.5309 - val_accuracy: 0.8045
Epoch 32/80
65/65 [==============================] - 16s 239ms/step - loss: 0.4029 - accuracy: 0.8589 - val_loss: 0.5251 - val_accuracy: 0.8102
Epoch 33/80
65/65 [==============================] - 15s 236ms/step - loss: 0.4019 - accuracy: 0.8570 - val_loss: 0.5250 - val_accuracy: 0.8045
Epoch 34/80
65/65 [==============================] - 15s 238ms/step - loss: 0.3761 - accuracy: 0.8692 - val_loss: 0.5314 - val_accuracy: 0.7955
Epoch 35/80
65/65 [==============================] - 16s 238ms/step - loss: 0.3824 - accuracy: 0.8638 - val_loss: 0.5108 - val_accuracy: 0.8136
Epoch 36/80
65/65 [==============================] - 16s 239ms/step - loss: 0.3793 - accuracy: 0.8628 - val_loss: 0.5216 - val_accuracy: 0.8136
Epoch 37/80
65/65 [==============================] - 16s 238ms/step - loss: 0.3634 - accuracy: 0.8774 - val_loss: 0.5584 - val_accuracy: 0.8000
Epoch 38/80
65/65 [==============================] - 16s 240ms/step - loss: 0.3785 - accuracy: 0.8706 - val_loss: 0.5148 - val_accuracy: 0.8205
Epoch 39/80
65/65 [==============================] - 15s 233ms/step - loss: 0.3710 - accuracy: 0.8696 - val_loss: 0.5166 - val_accuracy: 0.8159
Epoch 40/80
65/65 [==============================] - 16s 241ms/step - loss: 0.3676 - accuracy: 0.8740 - val_loss: 0.5369 - val_accuracy: 0.8080
Epoch 41/80
65/65 [==============================] - 16s 239ms/step - loss: 0.3605 - accuracy: 0.8755 - val_loss: 0.5122 - val_accuracy: 0.8136
Epoch 42/80
65/65 [==============================] - 16s 239ms/step - loss: 0.3738 - accuracy: 0.8716 - val_loss: 0.5113 - val_accuracy: 0.8068
Epoch 43/80
65/65 [==============================] - 16s 238ms/step - loss: 0.3670 - accuracy: 0.8662 - val_loss: 0.5173 - val_accuracy: 0.7977
Epoch 44/80
65/65 [==============================] - 16s 240ms/step - loss: 0.3556 - accuracy: 0.8808 - val_loss: 0.5117 - val_accuracy: 0.8114
Epoch 45/80
65/65 [==============================] - 16s 241ms/step - loss: 0.3402 - accuracy: 0.8847 - val_loss: 0.5001 - val_accuracy: 0.8182
Epoch 46/80
65/65 [==============================] - 16s 241ms/step - loss: 0.3496 - accuracy: 0.8823 - val_loss: 0.5126 - val_accuracy: 0.8023
Epoch 47/80
65/65 [==============================] - 16s 240ms/step - loss: 0.3330 - accuracy: 0.8842 - val_loss: 0.4949 - val_accuracy: 0.8205
Epoch 48/80
65/65 [==============================] - 16s 241ms/step - loss: 0.3407 - accuracy: 0.8915 - val_loss: 0.5089 - val_accuracy: 0.8125
Epoch 49/80
65/65 [==============================] - 16s 241ms/step - loss: 0.3272 - accuracy: 0.8857 - val_loss: 0.4977 - val_accuracy: 0.8216
Epoch 50/80
65/65 [==============================] - 16s 241ms/step - loss: 0.3334 - accuracy: 0.8906 - val_loss: 0.5078 - val_accuracy: 0.8182
Epoch 51/80
65/65 [==============================] - 16s 241ms/step - loss: 0.3226 - accuracy: 0.8891 - val_loss: 0.4885 - val_accuracy: 0.8182
Epoch 52/80
65/65 [==============================] - 16s 241ms/step - loss: 0.3411 - accuracy: 0.8804 - val_loss: 0.5080 - val_accuracy: 0.8125
Epoch 53/80
65/65 [==============================] - 16s 240ms/step - loss: 0.3239 - accuracy: 0.8959 - val_loss: 0.4837 - val_accuracy: 0.8273
Epoch 54/80
65/65 [==============================] - 16s 241ms/step - loss: 0.3189 - accuracy: 0.8886 - val_loss: 0.4995 - val_accuracy: 0.8068
Epoch 55/80
65/65 [==============================] - 16s 241ms/step - loss: 0.3213 - accuracy: 0.8901 - val_loss: 0.4823 - val_accuracy: 0.8227
Epoch 56/80
65/65 [==============================] - 16s 239ms/step - loss: 0.3246 - accuracy: 0.8867 - val_loss: 0.4830 - val_accuracy: 0.8261
Epoch 57/80
65/65 [==============================] - 16s 241ms/step - loss: 0.3214 - accuracy: 0.8979 - val_loss: 0.4930 - val_accuracy: 0.8295
Epoch 58/80
65/65 [==============================] - 16s 241ms/step - loss: 0.3093 - accuracy: 0.8954 - val_loss: 0.4909 - val_accuracy: 0.8341
Epoch 59/80
65/65 [==============================] - 16s 241ms/step - loss: 0.3019 - accuracy: 0.9003 - val_loss: 0.5005 - val_accuracy: 0.8148
Epoch 60/80
65/65 [==============================] - 16s 240ms/step - loss: 0.2890 - accuracy: 0.9090 - val_loss: 0.4927 - val_accuracy: 0.8182
Epoch 61/80
65/65 [==============================] - 16s 241ms/step - loss: 0.2967 - accuracy: 0.9013 - val_loss: 0.4960 - val_accuracy: 0.8136
Epoch 62/80
65/65 [==============================] - 16s 241ms/step - loss: 0.3067 - accuracy: 0.9008 - val_loss: 0.4869 - val_accuracy: 0.8193
Epoch 63/80
65/65 [==============================] - 16s 241ms/step - loss: 0.3004 - accuracy: 0.9052 - val_loss: 0.4937 - val_accuracy: 0.8205
Epoch 64/80
65/65 [==============================] - 16s 241ms/step - loss: 0.2983 - accuracy: 0.8945 - val_loss: 0.4778 - val_accuracy: 0.8307
Epoch 65/80
65/65 [==============================] - 16s 241ms/step - loss: 0.3093 - accuracy: 0.8876 - val_loss: 0.4978 - val_accuracy: 0.8239
Epoch 66/80
65/65 [==============================] - 16s 241ms/step - loss: 0.2971 - accuracy: 0.9066 - val_loss: 0.4856 - val_accuracy: 0.8250
Epoch 67/80
65/65 [==============================] - 16s 241ms/step - loss: 0.2988 - accuracy: 0.8998 - val_loss: 0.4988 - val_accuracy: 0.8091
Epoch 68/80
65/65 [==============================] - 16s 241ms/step - loss: 0.2979 - accuracy: 0.8974 - val_loss: 0.4810 - val_accuracy: 0.8307
Epoch 69/80
65/65 [==============================] - 16s 241ms/step - loss: 0.2958 - accuracy: 0.9027 - val_loss: 0.4811 - val_accuracy: 0.8239
Epoch 70/80
65/65 [==============================] - 16s 238ms/step - loss: 0.2924 - accuracy: 0.9052 - val_loss: 0.4825 - val_accuracy: 0.8239
Epoch 71/80
65/65 [==============================] - 16s 241ms/step - loss: 0.2946 - accuracy: 0.8993 - val_loss: 0.4774 - val_accuracy: 0.8250
Epoch 72/80
65/65 [==============================] - 16s 241ms/step - loss: 0.2937 - accuracy: 0.9032 - val_loss: 0.4898 - val_accuracy: 0.8136
Epoch 73/80
65/65 [==============================] - 16s 241ms/step - loss: 0.2844 - accuracy: 0.9032 - val_loss: 0.4707 - val_accuracy: 0.8284
Epoch 74/80
65/65 [==============================] - 16s 241ms/step - loss: 0.2905 - accuracy: 0.9032 - val_loss: 0.4793 - val_accuracy: 0.8273
Epoch 75/80
65/65 [==============================] - 16s 241ms/step - loss: 0.2728 - accuracy: 0.9134 - val_loss: 0.4627 - val_accuracy: 0.8386
Epoch 76/80
65/65 [==============================] - 16s 241ms/step - loss: 0.2777 - accuracy: 0.9086 - val_loss: 0.4730 - val_accuracy: 0.8284
Epoch 77/80
65/65 [==============================] - 16s 241ms/step - loss: 0.2890 - accuracy: 0.9071 - val_loss: 0.4771 - val_accuracy: 0.8250
Epoch 78/80
65/65 [==============================] - 16s 241ms/step - loss: 0.2703 - accuracy: 0.9163 - val_loss: 0.4920 - val_accuracy: 0.8091
Epoch 79/80
65/65 [==============================] - 16s 241ms/step - loss: 0.2811 - accuracy: 0.9095 - val_loss: 0.4670 - val_accuracy: 0.8284
Epoch 80/80
65/65 [==============================] - 16s 242ms/step - loss: 0.2726 - accuracy: 0.9154 - val_loss: 0.4727 - val_accuracy: 0.8386
In [ ]:
# Visualization of the learning curves

plt.subplot(1, 2, 1)
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.ylim([0.0, 2.0])
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.legend(['train', 'valid'])

plt.subplot(1, 2, 2)
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.ylim([0.5, 1.0])
plt.xlabel('Epoch')
plt.ylabel('Accuracy')
plt.legend(['train', 'valid'])

<matplotlib.legend.Legend at 0x230225ef970>
In [ ]:
net.save("models/B3DropRMS.keras")

This combination of Dropout layers reduces the overfitting without affecting validation accuracy. There is a bit of a gap in train and validation accuracy, but the two curves do not diverge, demonstrating that, even if the model has better performances on train than on validation, the increase in train accuracy brings an increase also in validation accuracy. Since this is one of the best models we have, we are going to test its accuracy on the test set.

In [ ]:
net = keras.models.load_model("models/B3DropRMS.keras")
net.evaluate(test_ds)

23/23 [==============================] - 4s 74ms/step - loss: 0.4477 - accuracy: 0.8365

[0.4477007985115051, 0.8365122675895691]

The test accuracy is of 83.65%. This is in line with the accuracy we had on validation set. We are going to compare this result with the one we achieve with a variant of the next combination.

Second Combination (0.1 + 0.2 + 0.3)¶

In [71]:
# Architecture definition
inputs = keras.Input((None,None,3))

x = inputs

x = keras.layers.RandomCrop(224, 224)(x)
x = keras.layers.RandomFlip(mode='horizontal')(x)
x = keras.layers.RandomContrast(0.25)(x)

x = keras.layers.Conv2D(32, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.MaxPooling2D(3, strides=3, padding="same")(x)
x = keras.layers.Dropout(0.1)(x)

x = keras.layers.Conv2D(64, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.MaxPooling2D(3, strides=3, padding="same")(x)
x = keras.layers.Dropout(0.2)(x)

x = keras.layers.Conv2D(128, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.Dropout(0.3)(x)

x = keras.layers.GlobalMaxPooling2D()(x)

outputs = keras.layers.Dense(num_classes, activation="softmax")(x)
net = keras.Model(inputs, outputs)
In [72]:
callback = keras.callbacks.EarlyStopping(monitor='val_loss', patience=10)
In [73]:
lr_schedule = keras.optimizers.schedules.InverseTimeDecay(
    initial_learning_rate = 0.0005,
    decay_steps = 65,
    decay_rate = 0.08)
optimizer = keras.optimizers.Adam(learning_rate=lr_schedule)
net.compile(loss=keras.losses.categorical_crossentropy,
            optimizer=optimizer,
            metrics=['accuracy'])
In [74]:
# Model training
history = net.fit(train_ds_256,
          epochs=80,
          validation_data=val_ds,
            callbacks=[callback])

Epoch 1/80
65/65 [==============================] - 8s 96ms/step - loss: 1.7872 - accuracy: 0.4509 - val_loss: 3.3091 - val_accuracy: 0.1841
Epoch 2/80
65/65 [==============================] - 6s 89ms/step - loss: 1.0530 - accuracy: 0.5875 - val_loss: 1.6237 - val_accuracy: 0.3193
Epoch 3/80
65/65 [==============================] - 6s 89ms/step - loss: 0.9619 - accuracy: 0.6304 - val_loss: 1.1959 - val_accuracy: 0.5341
Epoch 4/80
65/65 [==============================] - 6s 90ms/step - loss: 0.9180 - accuracy: 0.6459 - val_loss: 1.0566 - val_accuracy: 0.6057
Epoch 5/80
65/65 [==============================] - 6s 92ms/step - loss: 0.8424 - accuracy: 0.6678 - val_loss: 0.9566 - val_accuracy: 0.6307
Epoch 6/80
65/65 [==============================] - 7s 104ms/step - loss: 0.8379 - accuracy: 0.6804 - val_loss: 0.9065 - val_accuracy: 0.6682
Epoch 7/80
65/65 [==============================] - 7s 108ms/step - loss: 0.7702 - accuracy: 0.7053 - val_loss: 0.9177 - val_accuracy: 0.6477
Epoch 8/80
65/65 [==============================] - 7s 109ms/step - loss: 0.7612 - accuracy: 0.7155 - val_loss: 0.8579 - val_accuracy: 0.6818
Epoch 9/80
65/65 [==============================] - 7s 108ms/step - loss: 0.7122 - accuracy: 0.7305 - val_loss: 0.8717 - val_accuracy: 0.6761
Epoch 10/80
65/65 [==============================] - 7s 109ms/step - loss: 0.7161 - accuracy: 0.7237 - val_loss: 0.8707 - val_accuracy: 0.6886
Epoch 11/80
65/65 [==============================] - 7s 109ms/step - loss: 0.7028 - accuracy: 0.7359 - val_loss: 0.7498 - val_accuracy: 0.7295
Epoch 12/80
65/65 [==============================] - 7s 109ms/step - loss: 0.7027 - accuracy: 0.7315 - val_loss: 0.7955 - val_accuracy: 0.7091
Epoch 13/80
65/65 [==============================] - 7s 109ms/step - loss: 0.6379 - accuracy: 0.7661 - val_loss: 0.7371 - val_accuracy: 0.7307
Epoch 14/80
65/65 [==============================] - 7s 109ms/step - loss: 0.6463 - accuracy: 0.7568 - val_loss: 0.7812 - val_accuracy: 0.7193
Epoch 15/80
65/65 [==============================] - 7s 109ms/step - loss: 0.6336 - accuracy: 0.7578 - val_loss: 0.7118 - val_accuracy: 0.7466
Epoch 16/80
65/65 [==============================] - 7s 109ms/step - loss: 0.6192 - accuracy: 0.7675 - val_loss: 0.6928 - val_accuracy: 0.7466
Epoch 17/80
65/65 [==============================] - 7s 109ms/step - loss: 0.6138 - accuracy: 0.7758 - val_loss: 0.7229 - val_accuracy: 0.7352
Epoch 18/80
65/65 [==============================] - 7s 110ms/step - loss: 0.5753 - accuracy: 0.7943 - val_loss: 0.7063 - val_accuracy: 0.7432
Epoch 19/80
65/65 [==============================] - 7s 109ms/step - loss: 0.5780 - accuracy: 0.7758 - val_loss: 0.6975 - val_accuracy: 0.7273
Epoch 20/80
65/65 [==============================] - 7s 109ms/step - loss: 0.5681 - accuracy: 0.7923 - val_loss: 0.6757 - val_accuracy: 0.7580
Epoch 21/80
65/65 [==============================] - 7s 110ms/step - loss: 0.5650 - accuracy: 0.7943 - val_loss: 0.7459 - val_accuracy: 0.7307
Epoch 22/80
65/65 [==============================] - 8s 122ms/step - loss: 0.5533 - accuracy: 0.7884 - val_loss: 0.7118 - val_accuracy: 0.7398
Epoch 23/80
65/65 [==============================] - 9s 130ms/step - loss: 0.5717 - accuracy: 0.7831 - val_loss: 0.6429 - val_accuracy: 0.7625
Epoch 24/80
65/65 [==============================] - 9s 142ms/step - loss: 0.5477 - accuracy: 0.7928 - val_loss: 0.6544 - val_accuracy: 0.7648
Epoch 25/80
65/65 [==============================] - 10s 153ms/step - loss: 0.5500 - accuracy: 0.7977 - val_loss: 0.6530 - val_accuracy: 0.7568
Epoch 26/80
65/65 [==============================] - 11s 161ms/step - loss: 0.5245 - accuracy: 0.8157 - val_loss: 0.6647 - val_accuracy: 0.7511
Epoch 27/80
65/65 [==============================] - 11s 167ms/step - loss: 0.5185 - accuracy: 0.8074 - val_loss: 0.6406 - val_accuracy: 0.7727
Epoch 28/80
65/65 [==============================] - 11s 172ms/step - loss: 0.5222 - accuracy: 0.8089 - val_loss: 0.6446 - val_accuracy: 0.7648
Epoch 29/80
65/65 [==============================] - 12s 179ms/step - loss: 0.5108 - accuracy: 0.8205 - val_loss: 0.6421 - val_accuracy: 0.7670
Epoch 30/80
65/65 [==============================] - 13s 197ms/step - loss: 0.5265 - accuracy: 0.8074 - val_loss: 0.6366 - val_accuracy: 0.7500
Epoch 31/80
65/65 [==============================] - 12s 191ms/step - loss: 0.5000 - accuracy: 0.8142 - val_loss: 0.6114 - val_accuracy: 0.7841
Epoch 32/80
65/65 [==============================] - 13s 198ms/step - loss: 0.4927 - accuracy: 0.8215 - val_loss: 0.6258 - val_accuracy: 0.7875
Epoch 33/80
65/65 [==============================] - 13s 200ms/step - loss: 0.5253 - accuracy: 0.8137 - val_loss: 0.6120 - val_accuracy: 0.7841
Epoch 34/80
65/65 [==============================] - 13s 204ms/step - loss: 0.5104 - accuracy: 0.8171 - val_loss: 0.6292 - val_accuracy: 0.7693
Epoch 35/80
65/65 [==============================] - 13s 206ms/step - loss: 0.5016 - accuracy: 0.8127 - val_loss: 0.6058 - val_accuracy: 0.7807
Epoch 36/80
65/65 [==============================] - 14s 219ms/step - loss: 0.4808 - accuracy: 0.8283 - val_loss: 0.6090 - val_accuracy: 0.7693
Epoch 37/80
65/65 [==============================] - 13s 205ms/step - loss: 0.4706 - accuracy: 0.8303 - val_loss: 0.6275 - val_accuracy: 0.7716
Epoch 38/80
65/65 [==============================] - 14s 217ms/step - loss: 0.4650 - accuracy: 0.8307 - val_loss: 0.6521 - val_accuracy: 0.7557
Epoch 39/80
65/65 [==============================] - 14s 218ms/step - loss: 0.4695 - accuracy: 0.8288 - val_loss: 0.5960 - val_accuracy: 0.7773
Epoch 40/80
65/65 [==============================] - 14s 216ms/step - loss: 0.4742 - accuracy: 0.8327 - val_loss: 0.6046 - val_accuracy: 0.7875
Epoch 41/80
65/65 [==============================] - 15s 236ms/step - loss: 0.4680 - accuracy: 0.8332 - val_loss: 0.5862 - val_accuracy: 0.7875
Epoch 42/80
65/65 [==============================] - 15s 233ms/step - loss: 0.4660 - accuracy: 0.8298 - val_loss: 0.5886 - val_accuracy: 0.7830
Epoch 43/80
65/65 [==============================] - 15s 226ms/step - loss: 0.4763 - accuracy: 0.8191 - val_loss: 0.5914 - val_accuracy: 0.7886
Epoch 44/80
65/65 [==============================] - 15s 226ms/step - loss: 0.4707 - accuracy: 0.8341 - val_loss: 0.5892 - val_accuracy: 0.7955
Epoch 45/80
65/65 [==============================] - 15s 237ms/step - loss: 0.4640 - accuracy: 0.8346 - val_loss: 0.5915 - val_accuracy: 0.7784
Epoch 46/80
65/65 [==============================] - 15s 231ms/step - loss: 0.4465 - accuracy: 0.8419 - val_loss: 0.6042 - val_accuracy: 0.7886
Epoch 47/80
65/65 [==============================] - 15s 231ms/step - loss: 0.4369 - accuracy: 0.8497 - val_loss: 0.5647 - val_accuracy: 0.7989
Epoch 48/80
65/65 [==============================] - 15s 229ms/step - loss: 0.4455 - accuracy: 0.8361 - val_loss: 0.5791 - val_accuracy: 0.7932
Epoch 49/80
65/65 [==============================] - 16s 240ms/step - loss: 0.4412 - accuracy: 0.8322 - val_loss: 0.5670 - val_accuracy: 0.8023
Epoch 50/80
65/65 [==============================] - 15s 230ms/step - loss: 0.4523 - accuracy: 0.8385 - val_loss: 0.5833 - val_accuracy: 0.7784
Epoch 51/80
65/65 [==============================] - 16s 242ms/step - loss: 0.4447 - accuracy: 0.8395 - val_loss: 0.5820 - val_accuracy: 0.7875
Epoch 52/80
65/65 [==============================] - 16s 242ms/step - loss: 0.4188 - accuracy: 0.8531 - val_loss: 0.5793 - val_accuracy: 0.7966
Epoch 53/80
65/65 [==============================] - 15s 233ms/step - loss: 0.4281 - accuracy: 0.8390 - val_loss: 0.6048 - val_accuracy: 0.7818
Epoch 54/80
65/65 [==============================] - 16s 245ms/step - loss: 0.4438 - accuracy: 0.8468 - val_loss: 0.5724 - val_accuracy: 0.7898
Epoch 55/80
65/65 [==============================] - 15s 237ms/step - loss: 0.4182 - accuracy: 0.8517 - val_loss: 0.5473 - val_accuracy: 0.8057
Epoch 56/80
65/65 [==============================] - 16s 246ms/step - loss: 0.4199 - accuracy: 0.8444 - val_loss: 0.5526 - val_accuracy: 0.8011
Epoch 57/80
65/65 [==============================] - 16s 239ms/step - loss: 0.4135 - accuracy: 0.8512 - val_loss: 0.5775 - val_accuracy: 0.7909
Epoch 58/80
65/65 [==============================] - 16s 245ms/step - loss: 0.4083 - accuracy: 0.8624 - val_loss: 0.5696 - val_accuracy: 0.7909
Epoch 59/80
65/65 [==============================] - 16s 244ms/step - loss: 0.4208 - accuracy: 0.8478 - val_loss: 0.5717 - val_accuracy: 0.7920
Epoch 60/80
65/65 [==============================] - 16s 246ms/step - loss: 0.4332 - accuracy: 0.8497 - val_loss: 0.5910 - val_accuracy: 0.7864
Epoch 61/80
65/65 [==============================] - 16s 246ms/step - loss: 0.4105 - accuracy: 0.8541 - val_loss: 0.5524 - val_accuracy: 0.7977
Epoch 62/80
65/65 [==============================] - 16s 245ms/step - loss: 0.4038 - accuracy: 0.8531 - val_loss: 0.5598 - val_accuracy: 0.7909
Epoch 63/80
65/65 [==============================] - 16s 246ms/step - loss: 0.3978 - accuracy: 0.8594 - val_loss: 0.5601 - val_accuracy: 0.7966
Epoch 64/80
65/65 [==============================] - 16s 246ms/step - loss: 0.4117 - accuracy: 0.8463 - val_loss: 0.5496 - val_accuracy: 0.7989
Epoch 65/80
65/65 [==============================] - 16s 246ms/step - loss: 0.4070 - accuracy: 0.8575 - val_loss: 0.5611 - val_accuracy: 0.7989
In [76]:
# Visualization of the learning curves

plt.subplot(1, 2, 1)
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.ylim([0.0, 2.0])
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.legend(['train', 'valid'])

plt.subplot(1, 2, 2)
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.ylim([0.5, 1.0])
plt.xlabel('Epoch')
plt.ylabel('Accuracy')
plt.legend(['train', 'valid'])
Out[76]:
<matplotlib.legend.Legend at 0x22ecaf7caf0>
In [77]:
net.save("models/B3DropAv2.keras")

This second combination has stronger regularization power. In this way, the gap between train and validation is further decreased, but this time it does affect validation accuracy. What we can do, though, is deepening further the architechture of the model, in the hope that we can achieve the same (or better) validation accuracy but with a lower gap between train and validation.

Further deepening the model¶

Additional Dense Layer¶

In [25]:
# Architecture definition
inputs = keras.Input((None,None,3))

x = inputs

x = keras.layers.RandomCrop(224, 224)(x)
x = keras.layers.RandomFlip(mode='horizontal')(x)
x = keras.layers.RandomContrast(0.25)(x)

x = keras.layers.Conv2D(32, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.MaxPooling2D(3, strides=3, padding="same")(x)
x = keras.layers.Dropout(0.1)(x)

x = keras.layers.Conv2D(64, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.MaxPooling2D(3, strides=3, padding="same")(x)
x = keras.layers.Dropout(0.2)(x)

x = keras.layers.Conv2D(128, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.Dropout(0.3)(x)

x = keras.layers.GlobalMaxPooling2D()(x)

outputs = keras.layers.Dense(1024, activation="softmax")(x)

outputs = keras.layers.Dense(num_classes, activation="softmax")(x)
net = keras.Model(inputs, outputs)
In [26]:
callback = keras.callbacks.EarlyStopping(monitor='val_loss', patience=10)
In [27]:
lr_schedule = keras.optimizers.schedules.InverseTimeDecay(
    initial_learning_rate = 0.0005,
    decay_steps = 65,
    decay_rate = 0.08)
optimizer = keras.optimizers.Adam(learning_rate=lr_schedule)
net.compile(loss=keras.losses.categorical_crossentropy,
            optimizer=optimizer,
            metrics=['accuracy'])
In [28]:
# Model training
history = net.fit(train_ds_256,
          epochs=80,
          validation_data=val_ds,
            callbacks=[callback])

Epoch 1/80
65/65 [==============================] - 8s 112ms/step - loss: 1.6127 - accuracy: 0.4455 - val_loss: 1.7215 - val_accuracy: 0.3443
Epoch 2/80
65/65 [==============================] - 7s 109ms/step - loss: 1.0411 - accuracy: 0.6002 - val_loss: 1.1926 - val_accuracy: 0.4784
Epoch 3/80
65/65 [==============================] - 7s 108ms/step - loss: 0.9358 - accuracy: 0.6167 - val_loss: 0.9347 - val_accuracy: 0.6250
Epoch 4/80
65/65 [==============================] - 7s 109ms/step - loss: 0.8748 - accuracy: 0.6571 - val_loss: 0.8851 - val_accuracy: 0.6614
Epoch 5/80
65/65 [==============================] - 7s 108ms/step - loss: 0.8485 - accuracy: 0.6639 - val_loss: 0.8678 - val_accuracy: 0.6795
Epoch 6/80
65/65 [==============================] - 7s 109ms/step - loss: 0.8018 - accuracy: 0.6960 - val_loss: 0.9080 - val_accuracy: 0.6409
Epoch 7/80
65/65 [==============================] - 8s 122ms/step - loss: 0.7544 - accuracy: 0.7077 - val_loss: 0.8347 - val_accuracy: 0.6818
Epoch 8/80
65/65 [==============================] - 12s 180ms/step - loss: 0.7341 - accuracy: 0.7169 - val_loss: 0.7649 - val_accuracy: 0.7125
Epoch 9/80
65/65 [==============================] - 15s 228ms/step - loss: 0.6959 - accuracy: 0.7296 - val_loss: 0.8440 - val_accuracy: 0.6830
Epoch 10/80
65/65 [==============================] - 16s 248ms/step - loss: 0.7005 - accuracy: 0.7310 - val_loss: 0.8153 - val_accuracy: 0.7045
Epoch 11/80
65/65 [==============================] - 16s 245ms/step - loss: 0.6794 - accuracy: 0.7432 - val_loss: 0.8106 - val_accuracy: 0.7125
Epoch 12/80
65/65 [==============================] - 16s 245ms/step - loss: 0.6508 - accuracy: 0.7510 - val_loss: 0.7777 - val_accuracy: 0.7091
Epoch 13/80
65/65 [==============================] - 16s 244ms/step - loss: 0.6385 - accuracy: 0.7573 - val_loss: 0.7747 - val_accuracy: 0.7216
Epoch 14/80
65/65 [==============================] - 16s 246ms/step - loss: 0.6546 - accuracy: 0.7446 - val_loss: 0.6844 - val_accuracy: 0.7545
Epoch 15/80
65/65 [==============================] - 16s 245ms/step - loss: 0.6329 - accuracy: 0.7505 - val_loss: 0.6782 - val_accuracy: 0.7523
Epoch 16/80
65/65 [==============================] - 16s 245ms/step - loss: 0.5911 - accuracy: 0.7855 - val_loss: 0.7048 - val_accuracy: 0.7318
Epoch 17/80
65/65 [==============================] - 16s 246ms/step - loss: 0.6196 - accuracy: 0.7646 - val_loss: 0.6955 - val_accuracy: 0.7409
Epoch 18/80
65/65 [==============================] - 16s 246ms/step - loss: 0.5884 - accuracy: 0.7768 - val_loss: 0.6922 - val_accuracy: 0.7443
Epoch 19/80
65/65 [==============================] - 16s 246ms/step - loss: 0.5752 - accuracy: 0.7772 - val_loss: 0.6784 - val_accuracy: 0.7602
Epoch 20/80
65/65 [==============================] - 16s 246ms/step - loss: 0.5749 - accuracy: 0.7884 - val_loss: 0.6450 - val_accuracy: 0.7648
Epoch 21/80
65/65 [==============================] - 16s 246ms/step - loss: 0.5595 - accuracy: 0.7884 - val_loss: 0.6366 - val_accuracy: 0.7682
Epoch 22/80
65/65 [==============================] - 16s 247ms/step - loss: 0.5590 - accuracy: 0.7889 - val_loss: 0.6425 - val_accuracy: 0.7648
Epoch 23/80
65/65 [==============================] - 16s 247ms/step - loss: 0.5329 - accuracy: 0.8040 - val_loss: 0.6275 - val_accuracy: 0.7795
Epoch 24/80
65/65 [==============================] - 16s 248ms/step - loss: 0.5468 - accuracy: 0.7991 - val_loss: 0.7022 - val_accuracy: 0.7432
Epoch 25/80
65/65 [==============================] - 16s 248ms/step - loss: 0.5199 - accuracy: 0.8040 - val_loss: 0.6436 - val_accuracy: 0.7830
Epoch 26/80
65/65 [==============================] - 16s 247ms/step - loss: 0.5244 - accuracy: 0.8079 - val_loss: 0.6282 - val_accuracy: 0.7682
Epoch 27/80
65/65 [==============================] - 16s 248ms/step - loss: 0.5307 - accuracy: 0.8054 - val_loss: 0.6757 - val_accuracy: 0.7500
Epoch 28/80
65/65 [==============================] - 16s 248ms/step - loss: 0.5205 - accuracy: 0.8074 - val_loss: 0.6276 - val_accuracy: 0.7670
Epoch 29/80
65/65 [==============================] - 16s 249ms/step - loss: 0.5065 - accuracy: 0.8103 - val_loss: 0.6162 - val_accuracy: 0.7773
Epoch 30/80
65/65 [==============================] - 16s 247ms/step - loss: 0.4912 - accuracy: 0.8225 - val_loss: 0.6014 - val_accuracy: 0.7886
Epoch 31/80
65/65 [==============================] - 16s 248ms/step - loss: 0.4677 - accuracy: 0.8273 - val_loss: 0.5913 - val_accuracy: 0.7909
Epoch 32/80
65/65 [==============================] - 16s 248ms/step - loss: 0.4940 - accuracy: 0.8093 - val_loss: 0.6860 - val_accuracy: 0.7398
Epoch 33/80
65/65 [==============================] - 16s 250ms/step - loss: 0.4890 - accuracy: 0.8239 - val_loss: 0.6120 - val_accuracy: 0.7750
Epoch 34/80
65/65 [==============================] - 16s 248ms/step - loss: 0.4872 - accuracy: 0.8108 - val_loss: 0.6058 - val_accuracy: 0.7818
Epoch 35/80
65/65 [==============================] - 16s 249ms/step - loss: 0.4851 - accuracy: 0.8264 - val_loss: 0.5959 - val_accuracy: 0.7807
Epoch 36/80
65/65 [==============================] - 16s 248ms/step - loss: 0.4840 - accuracy: 0.8220 - val_loss: 0.6069 - val_accuracy: 0.7841
Epoch 37/80
65/65 [==============================] - 17s 251ms/step - loss: 0.4804 - accuracy: 0.8186 - val_loss: 0.6111 - val_accuracy: 0.7761
Epoch 38/80
65/65 [==============================] - 16s 249ms/step - loss: 0.4814 - accuracy: 0.8225 - val_loss: 0.6003 - val_accuracy: 0.7795
Epoch 39/80
65/65 [==============================] - 16s 249ms/step - loss: 0.4823 - accuracy: 0.8249 - val_loss: 0.5994 - val_accuracy: 0.7898
Epoch 40/80
65/65 [==============================] - 16s 250ms/step - loss: 0.4491 - accuracy: 0.8410 - val_loss: 0.6034 - val_accuracy: 0.7830
Epoch 41/80
65/65 [==============================] - 16s 250ms/step - loss: 0.4629 - accuracy: 0.8337 - val_loss: 0.6150 - val_accuracy: 0.7648
In [29]:
# Visualization of the learning curves

plt.subplot(1, 2, 1)
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.ylim([0.0, 2.0])
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.legend(['train', 'valid'])

plt.subplot(1, 2, 2)
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.ylim([0.5, 1.0])
plt.xlabel('Epoch')
plt.ylabel('Accuracy')
plt.legend(['train', 'valid'])
Out[29]:
<matplotlib.legend.Legend at 0x21608c17580>

An additional dense layer of 1024 further reduces the gap between train and validation.

Changing the Convolutional Layer¶

We are going to try and remove the first convolutional layer of 32 neuron while adding a last layer of 256.

In [39]:
# Architecture definition
inputs = keras.Input((None,None,3))

x = inputs

x = keras.layers.RandomCrop(224, 224)(x)
x = keras.layers.RandomFlip(mode='horizontal')(x)
x = keras.layers.RandomContrast(0.25)(x)

x = keras.layers.Conv2D(64, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.MaxPooling2D(3, strides=3, padding="same")(x)
x = keras.layers.Dropout(0.1)(x)

x = keras.layers.Conv2D(128, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.MaxPooling2D(3, strides=3, padding="same")(x)
x = keras.layers.Dropout(0.2)(x)

x = keras.layers.Conv2D(256, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.Dropout(0.3)(x)

x = keras.layers.GlobalMaxPooling2D()(x)

outputs = keras.layers.Dense(1024, activation="softmax")(x)

outputs = keras.layers.Dense(num_classes, activation="softmax")(x)
net = keras.Model(inputs, outputs)
In [40]:
callback = keras.callbacks.EarlyStopping(monitor='val_loss', patience=10)
In [41]:
lr_schedule = keras.optimizers.schedules.InverseTimeDecay(
    initial_learning_rate = 0.0005,
    decay_steps = 65,
    decay_rate = 0.08)
optimizer = keras.optimizers.Adam(learning_rate=lr_schedule)
net.compile(loss=keras.losses.categorical_crossentropy,
            optimizer=optimizer,
            metrics=['accuracy'])
In [42]:
# Model training
history = net.fit(train_ds_256,
          epochs=80,
          validation_data=val_ds,
            callbacks=[callback])

Epoch 1/80
65/65 [==============================] - 43s 529ms/step - loss: 1.5438 - accuracy: 0.4874 - val_loss: 1.6398 - val_accuracy: 0.3818
Epoch 2/80
65/65 [==============================] - 31s 482ms/step - loss: 1.0108 - accuracy: 0.6172 - val_loss: 0.9065 - val_accuracy: 0.6602
Epoch 3/80
65/65 [==============================] - 31s 483ms/step - loss: 0.8682 - accuracy: 0.6790 - val_loss: 0.9420 - val_accuracy: 0.6102
Epoch 4/80
65/65 [==============================] - 31s 483ms/step - loss: 0.7686 - accuracy: 0.6975 - val_loss: 0.7677 - val_accuracy: 0.7273
Epoch 5/80
65/65 [==============================] - 32s 483ms/step - loss: 0.7064 - accuracy: 0.7388 - val_loss: 0.7310 - val_accuracy: 0.7205
Epoch 6/80
65/65 [==============================] - 32s 486ms/step - loss: 0.6821 - accuracy: 0.7378 - val_loss: 0.6750 - val_accuracy: 0.7568
Epoch 7/80
65/65 [==============================] - 32s 485ms/step - loss: 0.6349 - accuracy: 0.7612 - val_loss: 0.7188 - val_accuracy: 0.7466
Epoch 8/80
65/65 [==============================] - 32s 486ms/step - loss: 0.5991 - accuracy: 0.7772 - val_loss: 0.7192 - val_accuracy: 0.7352
Epoch 9/80
65/65 [==============================] - 32s 486ms/step - loss: 0.5791 - accuracy: 0.7782 - val_loss: 0.7442 - val_accuracy: 0.7159
Epoch 10/80
65/65 [==============================] - 32s 485ms/step - loss: 0.5598 - accuracy: 0.7996 - val_loss: 0.7725 - val_accuracy: 0.6932
Epoch 11/80
65/65 [==============================] - 32s 486ms/step - loss: 0.5097 - accuracy: 0.8118 - val_loss: 0.6366 - val_accuracy: 0.7670
Epoch 12/80
65/65 [==============================] - 32s 488ms/step - loss: 0.5256 - accuracy: 0.8064 - val_loss: 0.6341 - val_accuracy: 0.7818
Epoch 13/80
65/65 [==============================] - 32s 489ms/step - loss: 0.5222 - accuracy: 0.8045 - val_loss: 0.5914 - val_accuracy: 0.7886
Epoch 14/80
65/65 [==============================] - 32s 488ms/step - loss: 0.4869 - accuracy: 0.8181 - val_loss: 0.6238 - val_accuracy: 0.7739
Epoch 15/80
65/65 [==============================] - 32s 489ms/step - loss: 0.4889 - accuracy: 0.8118 - val_loss: 0.5819 - val_accuracy: 0.7920
Epoch 16/80
65/65 [==============================] - 32s 487ms/step - loss: 0.4641 - accuracy: 0.8327 - val_loss: 0.5832 - val_accuracy: 0.8011
Epoch 17/80
65/65 [==============================] - 32s 486ms/step - loss: 0.4458 - accuracy: 0.8390 - val_loss: 0.5856 - val_accuracy: 0.7898
Epoch 18/80
65/65 [==============================] - 32s 485ms/step - loss: 0.4241 - accuracy: 0.8560 - val_loss: 0.5621 - val_accuracy: 0.8000
Epoch 19/80
65/65 [==============================] - 32s 493ms/step - loss: 0.4369 - accuracy: 0.8341 - val_loss: 0.5798 - val_accuracy: 0.7909
Epoch 20/80
65/65 [==============================] - 32s 489ms/step - loss: 0.4139 - accuracy: 0.8487 - val_loss: 0.6356 - val_accuracy: 0.7557
Epoch 21/80
65/65 [==============================] - 32s 488ms/step - loss: 0.4220 - accuracy: 0.8487 - val_loss: 0.5574 - val_accuracy: 0.7886
Epoch 22/80
65/65 [==============================] - 32s 489ms/step - loss: 0.3938 - accuracy: 0.8541 - val_loss: 0.5794 - val_accuracy: 0.7830
Epoch 23/80
65/65 [==============================] - 32s 487ms/step - loss: 0.4151 - accuracy: 0.8453 - val_loss: 0.5484 - val_accuracy: 0.8148
Epoch 24/80
65/65 [==============================] - 32s 486ms/step - loss: 0.3698 - accuracy: 0.8716 - val_loss: 0.5480 - val_accuracy: 0.8000
Epoch 25/80
65/65 [==============================] - 32s 488ms/step - loss: 0.3724 - accuracy: 0.8696 - val_loss: 0.5409 - val_accuracy: 0.8034
Epoch 26/80
65/65 [==============================] - 32s 487ms/step - loss: 0.3638 - accuracy: 0.8765 - val_loss: 0.5397 - val_accuracy: 0.8068
Epoch 27/80
65/65 [==============================] - 32s 489ms/step - loss: 0.3672 - accuracy: 0.8682 - val_loss: 0.5765 - val_accuracy: 0.7943
Epoch 28/80
65/65 [==============================] - 32s 486ms/step - loss: 0.3615 - accuracy: 0.8755 - val_loss: 0.5475 - val_accuracy: 0.8023
Epoch 29/80
65/65 [==============================] - 32s 490ms/step - loss: 0.3476 - accuracy: 0.8706 - val_loss: 0.5396 - val_accuracy: 0.8114
Epoch 30/80
65/65 [==============================] - 32s 489ms/step - loss: 0.3382 - accuracy: 0.8794 - val_loss: 0.5152 - val_accuracy: 0.8170
Epoch 31/80
65/65 [==============================] - 32s 487ms/step - loss: 0.3457 - accuracy: 0.8755 - val_loss: 0.5228 - val_accuracy: 0.8068
Epoch 32/80
65/65 [==============================] - 32s 486ms/step - loss: 0.3335 - accuracy: 0.8828 - val_loss: 0.5105 - val_accuracy: 0.8227
Epoch 33/80
65/65 [==============================] - 32s 487ms/step - loss: 0.3127 - accuracy: 0.9022 - val_loss: 0.5113 - val_accuracy: 0.8091
Epoch 34/80
65/65 [==============================] - 32s 487ms/step - loss: 0.3163 - accuracy: 0.8911 - val_loss: 0.5084 - val_accuracy: 0.8205
Epoch 35/80
65/65 [==============================] - 32s 487ms/step - loss: 0.3111 - accuracy: 0.8949 - val_loss: 0.5017 - val_accuracy: 0.8261
Epoch 36/80
65/65 [==============================] - 32s 486ms/step - loss: 0.3052 - accuracy: 0.8925 - val_loss: 0.5069 - val_accuracy: 0.8182
Epoch 37/80
65/65 [==============================] - 32s 487ms/step - loss: 0.2901 - accuracy: 0.9061 - val_loss: 0.5023 - val_accuracy: 0.8068
Epoch 38/80
65/65 [==============================] - 32s 487ms/step - loss: 0.2966 - accuracy: 0.9008 - val_loss: 0.4951 - val_accuracy: 0.8182
Epoch 39/80
65/65 [==============================] - 32s 487ms/step - loss: 0.2883 - accuracy: 0.9032 - val_loss: 0.5355 - val_accuracy: 0.8091
Epoch 40/80
65/65 [==============================] - 32s 486ms/step - loss: 0.2909 - accuracy: 0.8998 - val_loss: 0.4939 - val_accuracy: 0.8193
Epoch 41/80
65/65 [==============================] - 32s 487ms/step - loss: 0.2785 - accuracy: 0.9061 - val_loss: 0.4965 - val_accuracy: 0.8284
Epoch 42/80
65/65 [==============================] - 32s 486ms/step - loss: 0.2674 - accuracy: 0.9066 - val_loss: 0.4925 - val_accuracy: 0.8182
Epoch 43/80
65/65 [==============================] - 32s 486ms/step - loss: 0.2683 - accuracy: 0.9115 - val_loss: 0.4917 - val_accuracy: 0.8284
Epoch 44/80
65/65 [==============================] - 32s 487ms/step - loss: 0.2648 - accuracy: 0.9120 - val_loss: 0.4875 - val_accuracy: 0.8261
Epoch 45/80
65/65 [==============================] - 32s 486ms/step - loss: 0.2731 - accuracy: 0.9115 - val_loss: 0.5000 - val_accuracy: 0.8284
Epoch 46/80
65/65 [==============================] - 32s 486ms/step - loss: 0.2749 - accuracy: 0.9061 - val_loss: 0.4856 - val_accuracy: 0.8182
Epoch 47/80
65/65 [==============================] - 32s 487ms/step - loss: 0.2605 - accuracy: 0.9071 - val_loss: 0.4911 - val_accuracy: 0.8239
Epoch 48/80
65/65 [==============================] - 32s 488ms/step - loss: 0.2519 - accuracy: 0.9212 - val_loss: 0.4900 - val_accuracy: 0.8216
Epoch 49/80
65/65 [==============================] - 32s 487ms/step - loss: 0.2587 - accuracy: 0.9154 - val_loss: 0.4992 - val_accuracy: 0.8148
Epoch 50/80
65/65 [==============================] - 32s 487ms/step - loss: 0.2447 - accuracy: 0.9163 - val_loss: 0.5090 - val_accuracy: 0.8148
Epoch 51/80
65/65 [==============================] - 32s 487ms/step - loss: 0.2472 - accuracy: 0.9183 - val_loss: 0.4843 - val_accuracy: 0.8273
Epoch 52/80
65/65 [==============================] - 32s 486ms/step - loss: 0.2556 - accuracy: 0.9154 - val_loss: 0.4456 - val_accuracy: 0.8443
Epoch 53/80
65/65 [==============================] - 32s 487ms/step - loss: 0.2661 - accuracy: 0.9105 - val_loss: 0.4673 - val_accuracy: 0.8420
Epoch 54/80
65/65 [==============================] - 32s 487ms/step - loss: 0.2342 - accuracy: 0.9270 - val_loss: 0.4754 - val_accuracy: 0.8318
Epoch 55/80
65/65 [==============================] - 32s 486ms/step - loss: 0.2502 - accuracy: 0.9217 - val_loss: 0.4667 - val_accuracy: 0.8364
Epoch 56/80
65/65 [==============================] - 32s 486ms/step - loss: 0.2399 - accuracy: 0.9275 - val_loss: 0.4610 - val_accuracy: 0.8364
Epoch 57/80
65/65 [==============================] - 32s 486ms/step - loss: 0.2356 - accuracy: 0.9304 - val_loss: 0.4735 - val_accuracy: 0.8182
Epoch 58/80
65/65 [==============================] - 32s 486ms/step - loss: 0.2222 - accuracy: 0.9343 - val_loss: 0.4568 - val_accuracy: 0.8398
Epoch 59/80
65/65 [==============================] - 32s 488ms/step - loss: 0.2315 - accuracy: 0.9256 - val_loss: 0.4681 - val_accuracy: 0.8239
Epoch 60/80
65/65 [==============================] - 32s 486ms/step - loss: 0.2203 - accuracy: 0.9319 - val_loss: 0.4489 - val_accuracy: 0.8398
Epoch 61/80
65/65 [==============================] - 32s 488ms/step - loss: 0.2250 - accuracy: 0.9270 - val_loss: 0.4526 - val_accuracy: 0.8295
Epoch 62/80
65/65 [==============================] - 32s 487ms/step - loss: 0.2243 - accuracy: 0.9212 - val_loss: 0.4584 - val_accuracy: 0.8295
In [43]:
# Visualization of the learning curves

plt.subplot(1, 2, 1)
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.ylim([0.0, 2.0])
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.legend(['train', 'valid'])

plt.subplot(1, 2, 2)
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.ylim([0.5, 1.0])
plt.xlabel('Epoch')
plt.ylabel('Accuracy')
plt.legend(['train', 'valid'])
Out[43]:
<matplotlib.legend.Legend at 0x21616f19940>
In [44]:
net.save("models/B64ADrop.keras")

This modification in the architechture does bring an improvement. Of course, since we have more neurons, we have (again) increased the gap between train and validation. After other testing, we came to this architechture: adding another dense layer of 512 and adding a Dropout layer to the first dense layer.

In [90]:
# Architecture definition
inputs = keras.Input((None,None,3))

x = inputs

x = keras.layers.RandomCrop(224, 224)(x)
x = keras.layers.RandomFlip(mode='horizontal')(x)
x = keras.layers.RandomContrast(0.25)(x)

x = keras.layers.Conv2D(64, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.MaxPooling2D(3, strides=3, padding="same")(x)
x = keras.layers.Dropout(0.1)(x)

x = keras.layers.Conv2D(128, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.MaxPooling2D(3, strides=3, padding="same")(x)
x = keras.layers.Dropout(0.2)(x)

x = keras.layers.Conv2D(256, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.Dropout(0.2)(x)

x = keras.layers.GlobalMaxPooling2D()(x)

outputs = keras.layers.Dense(1024, activation="softmax")(x)
x = keras.layers.Dropout(0.1)(x)

outputs = keras.layers.Dense(512, activation="softmax")(x)

outputs = keras.layers.Dense(num_classes, activation="softmax")(x)
net = keras.Model(inputs, outputs)
In [91]:
callback = keras.callbacks.EarlyStopping(monitor='val_loss', patience=10)
In [92]:
lr_schedule = keras.optimizers.schedules.InverseTimeDecay(
    initial_learning_rate = 0.0005,
    decay_steps = 65,
    decay_rate = 0.08)
optimizer = keras.optimizers.Adam(learning_rate=lr_schedule)
net.compile(loss=keras.losses.categorical_crossentropy,
            optimizer=optimizer,
            metrics=['accuracy'])
In [93]:
# Model training
history = net.fit(train_ds_256,
          epochs=80,
          validation_data=val_ds,
            callbacks=[callback])

Epoch 1/80
65/65 [==============================] - 14s 183ms/step - loss: 1.8859 - accuracy: 0.4703 - val_loss: 3.2992 - val_accuracy: 0.3000
Epoch 2/80
65/65 [==============================] - 13s 198ms/step - loss: 1.2943 - accuracy: 0.5696 - val_loss: 1.2053 - val_accuracy: 0.5170
Epoch 3/80
65/65 [==============================] - 13s 198ms/step - loss: 1.1326 - accuracy: 0.5968 - val_loss: 0.9673 - val_accuracy: 0.6455
Epoch 4/80
65/65 [==============================] - 13s 198ms/step - loss: 1.0899 - accuracy: 0.6172 - val_loss: 0.7542 - val_accuracy: 0.7216
Epoch 5/80
65/65 [==============================] - 13s 200ms/step - loss: 0.9838 - accuracy: 0.6459 - val_loss: 0.7256 - val_accuracy: 0.7182
Epoch 6/80
65/65 [==============================] - 19s 290ms/step - loss: 0.8945 - accuracy: 0.6775 - val_loss: 0.7948 - val_accuracy: 0.6943
Epoch 7/80
65/65 [==============================] - 30s 470ms/step - loss: 0.8563 - accuracy: 0.6960 - val_loss: 0.8255 - val_accuracy: 0.6830
Epoch 8/80
65/65 [==============================] - 31s 479ms/step - loss: 0.8284 - accuracy: 0.7038 - val_loss: 0.8459 - val_accuracy: 0.6943
Epoch 9/80
65/65 [==============================] - 31s 482ms/step - loss: 0.8205 - accuracy: 0.7004 - val_loss: 0.6965 - val_accuracy: 0.7443
Epoch 10/80
65/65 [==============================] - 31s 483ms/step - loss: 0.7780 - accuracy: 0.7130 - val_loss: 0.6953 - val_accuracy: 0.7455
Epoch 11/80
65/65 [==============================] - 31s 482ms/step - loss: 0.7269 - accuracy: 0.7330 - val_loss: 0.7213 - val_accuracy: 0.7375
Epoch 12/80
65/65 [==============================] - 31s 483ms/step - loss: 0.7273 - accuracy: 0.7378 - val_loss: 0.6059 - val_accuracy: 0.7739
Epoch 13/80
65/65 [==============================] - 32s 484ms/step - loss: 0.7469 - accuracy: 0.7262 - val_loss: 0.6504 - val_accuracy: 0.7602
Epoch 14/80
65/65 [==============================] - 32s 484ms/step - loss: 0.7115 - accuracy: 0.7335 - val_loss: 0.7031 - val_accuracy: 0.7523
Epoch 15/80
65/65 [==============================] - 32s 484ms/step - loss: 0.6672 - accuracy: 0.7578 - val_loss: 0.6071 - val_accuracy: 0.7807
Epoch 16/80
65/65 [==============================] - 32s 484ms/step - loss: 0.7046 - accuracy: 0.7344 - val_loss: 0.6054 - val_accuracy: 0.7830
Epoch 17/80
65/65 [==============================] - 32s 485ms/step - loss: 0.6495 - accuracy: 0.7617 - val_loss: 0.6571 - val_accuracy: 0.7580
Epoch 18/80
65/65 [==============================] - 32s 484ms/step - loss: 0.6569 - accuracy: 0.7631 - val_loss: 0.6189 - val_accuracy: 0.7716
Epoch 19/80
65/65 [==============================] - 32s 485ms/step - loss: 0.6486 - accuracy: 0.7622 - val_loss: 0.6332 - val_accuracy: 0.7705
Epoch 20/80
65/65 [==============================] - 32s 485ms/step - loss: 0.6259 - accuracy: 0.7597 - val_loss: 0.5570 - val_accuracy: 0.7852
Epoch 21/80
65/65 [==============================] - 32s 485ms/step - loss: 0.6037 - accuracy: 0.7719 - val_loss: 0.5820 - val_accuracy: 0.7864
Epoch 22/80
65/65 [==============================] - 32s 485ms/step - loss: 0.6138 - accuracy: 0.7840 - val_loss: 0.5729 - val_accuracy: 0.8000
Epoch 23/80
65/65 [==============================] - 32s 485ms/step - loss: 0.5609 - accuracy: 0.8030 - val_loss: 0.6272 - val_accuracy: 0.7705
Epoch 24/80
65/65 [==============================] - 32s 485ms/step - loss: 0.5776 - accuracy: 0.7850 - val_loss: 0.6072 - val_accuracy: 0.7670
Epoch 25/80
65/65 [==============================] - 32s 485ms/step - loss: 0.5860 - accuracy: 0.7928 - val_loss: 0.5671 - val_accuracy: 0.7909
Epoch 26/80
65/65 [==============================] - 32s 486ms/step - loss: 0.5879 - accuracy: 0.7768 - val_loss: 0.5958 - val_accuracy: 0.7818
Epoch 27/80
65/65 [==============================] - 32s 486ms/step - loss: 0.5704 - accuracy: 0.7986 - val_loss: 0.5407 - val_accuracy: 0.7955
Epoch 28/80
65/65 [==============================] - 32s 485ms/step - loss: 0.5207 - accuracy: 0.8050 - val_loss: 0.5266 - val_accuracy: 0.8068
Epoch 29/80
65/65 [==============================] - 32s 486ms/step - loss: 0.5785 - accuracy: 0.7962 - val_loss: 0.5195 - val_accuracy: 0.8080
Epoch 30/80
65/65 [==============================] - 32s 486ms/step - loss: 0.5412 - accuracy: 0.8035 - val_loss: 0.5055 - val_accuracy: 0.8148
Epoch 31/80
65/65 [==============================] - 32s 486ms/step - loss: 0.5248 - accuracy: 0.8006 - val_loss: 0.5657 - val_accuracy: 0.7875
Epoch 32/80
65/65 [==============================] - 32s 486ms/step - loss: 0.5053 - accuracy: 0.8176 - val_loss: 0.5376 - val_accuracy: 0.8045
Epoch 33/80
65/65 [==============================] - 32s 486ms/step - loss: 0.5336 - accuracy: 0.8011 - val_loss: 0.5257 - val_accuracy: 0.8125
Epoch 34/80
65/65 [==============================] - 32s 486ms/step - loss: 0.4949 - accuracy: 0.8196 - val_loss: 0.5597 - val_accuracy: 0.7920
Epoch 35/80
65/65 [==============================] - 32s 486ms/step - loss: 0.5130 - accuracy: 0.7996 - val_loss: 0.4878 - val_accuracy: 0.8284
Epoch 36/80
65/65 [==============================] - 32s 486ms/step - loss: 0.4793 - accuracy: 0.8186 - val_loss: 0.5223 - val_accuracy: 0.8045
Epoch 37/80
65/65 [==============================] - 32s 486ms/step - loss: 0.4737 - accuracy: 0.8225 - val_loss: 0.5287 - val_accuracy: 0.8045
Epoch 38/80
65/65 [==============================] - 32s 486ms/step - loss: 0.4954 - accuracy: 0.8103 - val_loss: 0.5251 - val_accuracy: 0.7966
Epoch 39/80
65/65 [==============================] - 32s 487ms/step - loss: 0.4840 - accuracy: 0.8157 - val_loss: 0.5587 - val_accuracy: 0.7932
Epoch 40/80
65/65 [==============================] - 32s 487ms/step - loss: 0.4855 - accuracy: 0.8147 - val_loss: 0.5365 - val_accuracy: 0.8080
Epoch 41/80
65/65 [==============================] - 32s 487ms/step - loss: 0.4871 - accuracy: 0.8259 - val_loss: 0.4964 - val_accuracy: 0.8205
Epoch 42/80
65/65 [==============================] - 32s 488ms/step - loss: 0.4625 - accuracy: 0.8200 - val_loss: 0.4985 - val_accuracy: 0.8091
Epoch 43/80
65/65 [==============================] - 32s 488ms/step - loss: 0.4572 - accuracy: 0.8303 - val_loss: 0.5099 - val_accuracy: 0.8216
Epoch 44/80
65/65 [==============================] - 32s 488ms/step - loss: 0.4539 - accuracy: 0.8337 - val_loss: 0.5253 - val_accuracy: 0.8045
Epoch 45/80
65/65 [==============================] - 32s 487ms/step - loss: 0.4627 - accuracy: 0.8356 - val_loss: 0.5123 - val_accuracy: 0.8114
In [94]:
# Visualization of the learning curves

plt.subplot(1, 2, 1)
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.ylim([0.0, 2.0])
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.legend(['train', 'valid'])

plt.subplot(1, 2, 2)
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.ylim([0.5, 1.0])
plt.xlabel('Epoch')
plt.ylabel('Accuracy')
plt.legend(['train', 'valid'])
Out[94]:
<matplotlib.legend.Legend at 0x215a7700820>

By decreasing the strength of the dropout layer, we could be able to let the model learn more:

In [16]:
# Architecture definition
inputs = keras.Input((None,None,3))

x = inputs

x = keras.layers.RandomCrop(224, 224)(x)
x = keras.layers.RandomFlip(mode='horizontal')(x)
x = keras.layers.RandomContrast(0.25)(x)

x = keras.layers.Conv2D(64, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.MaxPooling2D(3, strides=3, padding="same")(x)
x = keras.layers.Dropout(0.1)(x)

x = keras.layers.Conv2D(128, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.MaxPooling2D(3, strides=3, padding="same")(x)
x = keras.layers.Dropout(0.1)(x)

x = keras.layers.Conv2D(256, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.Dropout(0.1)(x)

x = keras.layers.GlobalMaxPooling2D()(x)

outputs = keras.layers.Dense(1024, activation="softmax")(x)
x = keras.layers.Dropout(0.2)(x)

outputs = keras.layers.Dense(512, activation="softmax")(x)

outputs = keras.layers.Dense(num_classes, activation="softmax")(x)
net = keras.Model(inputs, outputs)
In [17]:
callback = keras.callbacks.EarlyStopping(monitor='val_loss', patience=10)
In [18]:
lr_schedule = keras.optimizers.schedules.InverseTimeDecay(
    initial_learning_rate = 0.0005,
    decay_steps = 65,
    decay_rate = 0.08)
optimizer = keras.optimizers.Adam(learning_rate=lr_schedule)
net.compile(loss=keras.losses.categorical_crossentropy,
            optimizer=optimizer,
            metrics=['accuracy'])
In [19]:
# Model training
history = net.fit(train_ds_256,
          epochs=80,
          validation_data=val_ds,
            callbacks=[callback])

Epoch 1/80
74/74 [==============================] - 20s 196ms/step - loss: 1.9844 - accuracy: 0.4551 - val_loss: 2.4820 - val_accuracy: 0.3646
Epoch 2/80
74/74 [==============================] - 12s 165ms/step - loss: 1.3621 - accuracy: 0.5645 - val_loss: 1.3708 - val_accuracy: 0.5468
Epoch 3/80
74/74 [==============================] - 14s 184ms/step - loss: 1.2446 - accuracy: 0.5824 - val_loss: 0.9081 - val_accuracy: 0.6661
Epoch 4/80
74/74 [==============================] - 14s 185ms/step - loss: 1.1093 - accuracy: 0.6301 - val_loss: 0.8527 - val_accuracy: 0.6644
Epoch 5/80
74/74 [==============================] - 14s 185ms/step - loss: 1.0274 - accuracy: 0.6356 - val_loss: 0.8337 - val_accuracy: 0.6985
Epoch 6/80
74/74 [==============================] - 14s 185ms/step - loss: 1.0037 - accuracy: 0.6564 - val_loss: 0.8142 - val_accuracy: 0.7189
Epoch 7/80
74/74 [==============================] - 14s 186ms/step - loss: 0.9306 - accuracy: 0.6701 - val_loss: 0.8056 - val_accuracy: 0.7223
Epoch 8/80
74/74 [==============================] - 14s 185ms/step - loss: 0.9002 - accuracy: 0.6790 - val_loss: 0.7858 - val_accuracy: 0.7155
Epoch 9/80
74/74 [==============================] - 14s 186ms/step - loss: 0.8724 - accuracy: 0.6943 - val_loss: 0.7612 - val_accuracy: 0.7223
Epoch 10/80
74/74 [==============================] - 14s 189ms/step - loss: 0.8404 - accuracy: 0.7024 - val_loss: 0.8242 - val_accuracy: 0.6814
Epoch 11/80
74/74 [==============================] - 15s 205ms/step - loss: 0.8075 - accuracy: 0.7097 - val_loss: 0.7140 - val_accuracy: 0.7257
Epoch 12/80
74/74 [==============================] - 19s 258ms/step - loss: 0.7972 - accuracy: 0.7033 - val_loss: 0.7650 - val_accuracy: 0.7070
Epoch 13/80
74/74 [==============================] - 32s 434ms/step - loss: 0.7726 - accuracy: 0.7169 - val_loss: 0.6636 - val_accuracy: 0.7342
Epoch 14/80
74/74 [==============================] - 33s 443ms/step - loss: 0.7656 - accuracy: 0.7135 - val_loss: 0.7408 - val_accuracy: 0.7359
Epoch 15/80
74/74 [==============================] - 29s 390ms/step - loss: 0.7856 - accuracy: 0.7143 - val_loss: 0.7319 - val_accuracy: 0.7240
Epoch 16/80
74/74 [==============================] - 32s 432ms/step - loss: 0.7181 - accuracy: 0.7267 - val_loss: 0.6348 - val_accuracy: 0.7649
Epoch 17/80
74/74 [==============================] - 34s 455ms/step - loss: 0.6919 - accuracy: 0.7531 - val_loss: 0.7148 - val_accuracy: 0.7445
Epoch 18/80
74/74 [==============================] - 34s 454ms/step - loss: 0.6770 - accuracy: 0.7518 - val_loss: 0.6076 - val_accuracy: 0.7734
Epoch 19/80
74/74 [==============================] - 34s 455ms/step - loss: 0.6906 - accuracy: 0.7382 - val_loss: 0.6340 - val_accuracy: 0.7751
Epoch 20/80
74/74 [==============================] - 34s 454ms/step - loss: 0.6754 - accuracy: 0.7467 - val_loss: 0.6440 - val_accuracy: 0.7547
Epoch 21/80
74/74 [==============================] - 34s 455ms/step - loss: 0.6702 - accuracy: 0.7458 - val_loss: 0.6200 - val_accuracy: 0.7632
Epoch 22/80
74/74 [==============================] - 34s 457ms/step - loss: 0.6738 - accuracy: 0.7480 - val_loss: 0.5940 - val_accuracy: 0.7853
Epoch 23/80
74/74 [==============================] - 34s 455ms/step - loss: 0.6334 - accuracy: 0.7671 - val_loss: 0.5580 - val_accuracy: 0.7853
Epoch 24/80
74/74 [==============================] - 34s 455ms/step - loss: 0.6414 - accuracy: 0.7710 - val_loss: 0.5683 - val_accuracy: 0.7888
Epoch 25/80
74/74 [==============================] - 34s 455ms/step - loss: 0.6289 - accuracy: 0.7799 - val_loss: 0.6002 - val_accuracy: 0.7785
Epoch 26/80
74/74 [==============================] - 34s 455ms/step - loss: 0.5742 - accuracy: 0.7905 - val_loss: 0.6284 - val_accuracy: 0.7666
Epoch 27/80
74/74 [==============================] - 34s 454ms/step - loss: 0.6165 - accuracy: 0.7756 - val_loss: 0.5915 - val_accuracy: 0.7785
Epoch 28/80
74/74 [==============================] - 34s 454ms/step - loss: 0.6174 - accuracy: 0.7693 - val_loss: 0.5421 - val_accuracy: 0.7922
Epoch 29/80
74/74 [==============================] - 34s 454ms/step - loss: 0.5481 - accuracy: 0.7944 - val_loss: 0.5401 - val_accuracy: 0.8109
Epoch 30/80
74/74 [==============================] - 34s 454ms/step - loss: 0.5843 - accuracy: 0.7910 - val_loss: 0.5518 - val_accuracy: 0.7871
Epoch 31/80
74/74 [==============================] - 34s 455ms/step - loss: 0.5620 - accuracy: 0.7867 - val_loss: 0.5149 - val_accuracy: 0.8092
Epoch 32/80
74/74 [==============================] - 34s 454ms/step - loss: 0.5748 - accuracy: 0.7944 - val_loss: 0.5708 - val_accuracy: 0.7939
Epoch 33/80
74/74 [==============================] - 34s 454ms/step - loss: 0.5504 - accuracy: 0.7850 - val_loss: 0.6118 - val_accuracy: 0.7802
Epoch 34/80
74/74 [==============================] - 34s 454ms/step - loss: 0.5628 - accuracy: 0.7888 - val_loss: 0.5052 - val_accuracy: 0.8092
Epoch 35/80
74/74 [==============================] - 34s 454ms/step - loss: 0.5564 - accuracy: 0.7969 - val_loss: 0.5953 - val_accuracy: 0.7853
Epoch 36/80
74/74 [==============================] - 34s 455ms/step - loss: 0.5528 - accuracy: 0.7923 - val_loss: 0.5994 - val_accuracy: 0.7888
Epoch 37/80
74/74 [==============================] - 34s 455ms/step - loss: 0.5300 - accuracy: 0.8118 - val_loss: 0.5234 - val_accuracy: 0.8160
Epoch 38/80
74/74 [==============================] - 34s 454ms/step - loss: 0.5220 - accuracy: 0.8135 - val_loss: 0.5333 - val_accuracy: 0.8109
Epoch 39/80
74/74 [==============================] - 34s 455ms/step - loss: 0.5078 - accuracy: 0.8050 - val_loss: 0.5911 - val_accuracy: 0.7905
Epoch 40/80
74/74 [==============================] - 34s 455ms/step - loss: 0.5376 - accuracy: 0.8076 - val_loss: 0.5477 - val_accuracy: 0.7836
Epoch 41/80
74/74 [==============================] - 34s 457ms/step - loss: 0.5082 - accuracy: 0.8050 - val_loss: 0.5029 - val_accuracy: 0.8041
Epoch 42/80
74/74 [==============================] - 34s 456ms/step - loss: 0.5141 - accuracy: 0.8042 - val_loss: 0.5414 - val_accuracy: 0.8075
Epoch 43/80
74/74 [==============================] - 34s 454ms/step - loss: 0.4895 - accuracy: 0.8123 - val_loss: 0.5246 - val_accuracy: 0.8075
Epoch 44/80
74/74 [==============================] - 34s 454ms/step - loss: 0.5343 - accuracy: 0.8123 - val_loss: 0.5642 - val_accuracy: 0.7922
Epoch 45/80
74/74 [==============================] - 34s 454ms/step - loss: 0.5052 - accuracy: 0.8140 - val_loss: 0.5084 - val_accuracy: 0.8143
Epoch 46/80
74/74 [==============================] - 34s 455ms/step - loss: 0.5415 - accuracy: 0.7952 - val_loss: 0.5637 - val_accuracy: 0.7973
Epoch 47/80
74/74 [==============================] - 34s 456ms/step - loss: 0.4939 - accuracy: 0.8229 - val_loss: 0.5383 - val_accuracy: 0.8109
Epoch 48/80
74/74 [==============================] - 34s 455ms/step - loss: 0.4718 - accuracy: 0.8203 - val_loss: 0.4880 - val_accuracy: 0.8245
Epoch 49/80
74/74 [==============================] - 34s 456ms/step - loss: 0.5012 - accuracy: 0.8089 - val_loss: 0.5435 - val_accuracy: 0.7990
Epoch 50/80
74/74 [==============================] - 34s 457ms/step - loss: 0.4789 - accuracy: 0.8186 - val_loss: 0.4766 - val_accuracy: 0.8330
Epoch 51/80
74/74 [==============================] - 34s 456ms/step - loss: 0.4702 - accuracy: 0.8246 - val_loss: 0.5048 - val_accuracy: 0.8143
Epoch 52/80
74/74 [==============================] - 34s 456ms/step - loss: 0.4763 - accuracy: 0.8276 - val_loss: 0.5325 - val_accuracy: 0.8075
Epoch 53/80
74/74 [==============================] - 34s 457ms/step - loss: 0.5007 - accuracy: 0.8135 - val_loss: 0.4931 - val_accuracy: 0.8143
Epoch 54/80
74/74 [==============================] - 34s 456ms/step - loss: 0.4764 - accuracy: 0.8165 - val_loss: 0.4932 - val_accuracy: 0.8126
Epoch 55/80
74/74 [==============================] - 34s 458ms/step - loss: 0.4799 - accuracy: 0.8323 - val_loss: 0.4713 - val_accuracy: 0.8296
Epoch 56/80
74/74 [==============================] - 34s 461ms/step - loss: 0.4630 - accuracy: 0.8323 - val_loss: 0.5393 - val_accuracy: 0.8109
Epoch 57/80
74/74 [==============================] - 34s 456ms/step - loss: 0.4511 - accuracy: 0.8284 - val_loss: 0.5037 - val_accuracy: 0.8211
Epoch 58/80
74/74 [==============================] - 34s 458ms/step - loss: 0.4497 - accuracy: 0.8293 - val_loss: 0.5124 - val_accuracy: 0.8143
Epoch 59/80
74/74 [==============================] - 34s 459ms/step - loss: 0.4836 - accuracy: 0.8199 - val_loss: 0.4896 - val_accuracy: 0.8211
Epoch 60/80
74/74 [==============================] - 34s 458ms/step - loss: 0.4422 - accuracy: 0.8365 - val_loss: 0.4836 - val_accuracy: 0.8160
Epoch 61/80
74/74 [==============================] - 34s 457ms/step - loss: 0.4249 - accuracy: 0.8378 - val_loss: 0.5015 - val_accuracy: 0.8160
Epoch 62/80
74/74 [==============================] - 34s 455ms/step - loss: 0.4239 - accuracy: 0.8455 - val_loss: 0.4825 - val_accuracy: 0.8228
Epoch 63/80
74/74 [==============================] - 34s 458ms/step - loss: 0.4614 - accuracy: 0.8306 - val_loss: 0.5102 - val_accuracy: 0.8160
Epoch 64/80
74/74 [==============================] - 34s 456ms/step - loss: 0.4431 - accuracy: 0.8348 - val_loss: 0.4949 - val_accuracy: 0.8160
Epoch 65/80
74/74 [==============================] - 34s 457ms/step - loss: 0.4297 - accuracy: 0.8378 - val_loss: 0.4692 - val_accuracy: 0.8313
Epoch 66/80
74/74 [==============================] - 34s 458ms/step - loss: 0.4329 - accuracy: 0.8378 - val_loss: 0.4937 - val_accuracy: 0.8228
Epoch 67/80
74/74 [==============================] - 34s 455ms/step - loss: 0.4148 - accuracy: 0.8455 - val_loss: 0.4974 - val_accuracy: 0.8245
Epoch 68/80
74/74 [==============================] - 34s 457ms/step - loss: 0.4551 - accuracy: 0.8370 - val_loss: 0.4953 - val_accuracy: 0.8211
Epoch 69/80
74/74 [==============================] - 34s 454ms/step - loss: 0.4448 - accuracy: 0.8301 - val_loss: 0.4762 - val_accuracy: 0.8279
Epoch 70/80
74/74 [==============================] - 34s 456ms/step - loss: 0.4316 - accuracy: 0.8382 - val_loss: 0.5186 - val_accuracy: 0.8075
Epoch 71/80
74/74 [==============================] - 34s 454ms/step - loss: 0.4135 - accuracy: 0.8442 - val_loss: 0.4732 - val_accuracy: 0.8416
Epoch 72/80
74/74 [==============================] - 34s 454ms/step - loss: 0.4464 - accuracy: 0.8374 - val_loss: 0.4651 - val_accuracy: 0.8313
Epoch 73/80
74/74 [==============================] - 34s 454ms/step - loss: 0.4416 - accuracy: 0.8357 - val_loss: 0.4681 - val_accuracy: 0.8365
Epoch 74/80
74/74 [==============================] - 34s 455ms/step - loss: 0.4474 - accuracy: 0.8348 - val_loss: 0.4427 - val_accuracy: 0.8484
Epoch 75/80
74/74 [==============================] - 34s 457ms/step - loss: 0.4234 - accuracy: 0.8429 - val_loss: 0.4529 - val_accuracy: 0.8399
Epoch 76/80
74/74 [==============================] - 34s 455ms/step - loss: 0.4029 - accuracy: 0.8561 - val_loss: 0.4713 - val_accuracy: 0.8262
Epoch 77/80
74/74 [==============================] - 34s 454ms/step - loss: 0.4278 - accuracy: 0.8446 - val_loss: 0.4788 - val_accuracy: 0.8194
Epoch 78/80
74/74 [==============================] - 34s 453ms/step - loss: 0.4035 - accuracy: 0.8480 - val_loss: 0.4780 - val_accuracy: 0.8177
Epoch 79/80
74/74 [==============================] - 34s 454ms/step - loss: 0.3988 - accuracy: 0.8472 - val_loss: 0.4661 - val_accuracy: 0.8296
Epoch 80/80
74/74 [==============================] - 34s 454ms/step - loss: 0.4322 - accuracy: 0.8425 - val_loss: 0.4736 - val_accuracy: 0.8330
In [20]:
# Visualization of the learning curves

plt.subplot(1, 2, 1)
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.ylim([0.0, 2.0])
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.legend(['train', 'valid'])

plt.subplot(1, 2, 2)
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.ylim([0.5, 1.0])
plt.xlabel('Epoch')
plt.ylabel('Accuracy')
plt.legend(['train', 'valid'])
Out[20]:
<matplotlib.legend.Legend at 0x257d6994850>
In [21]:
net.save("models/B3D2DropA.keras")

We achieved a model with zero gap between train and validation accuracy, while retaining nearly the same validation accuracy as the first model we tested. Since the early stopping did not stop the training of the model, we are going to let it train for another bit:

In [23]:
# Train until the early stopping blocks it
# Model training
history = net.fit(train_ds_256,
          epochs=80,
          validation_data=val_ds,
            callbacks=[callback])

Epoch 1/80
74/74 [==============================] - 11s 148ms/step - loss: 0.4120 - accuracy: 0.8506 - val_loss: 0.4953 - val_accuracy: 0.8109
Epoch 2/80
74/74 [==============================] - 11s 150ms/step - loss: 0.4338 - accuracy: 0.8318 - val_loss: 0.4753 - val_accuracy: 0.8279
Epoch 3/80
74/74 [==============================] - 12s 158ms/step - loss: 0.4116 - accuracy: 0.8484 - val_loss: 0.4669 - val_accuracy: 0.8296
Epoch 4/80
74/74 [==============================] - 14s 183ms/step - loss: 0.3943 - accuracy: 0.8523 - val_loss: 0.4768 - val_accuracy: 0.8296
Epoch 5/80
74/74 [==============================] - 14s 184ms/step - loss: 0.3898 - accuracy: 0.8595 - val_loss: 0.4913 - val_accuracy: 0.8143
Epoch 6/80
74/74 [==============================] - 14s 184ms/step - loss: 0.3964 - accuracy: 0.8446 - val_loss: 0.4813 - val_accuracy: 0.8245
Epoch 7/80
74/74 [==============================] - 14s 183ms/step - loss: 0.3874 - accuracy: 0.8599 - val_loss: 0.4444 - val_accuracy: 0.8313
Epoch 8/80
74/74 [==============================] - 14s 184ms/step - loss: 0.3964 - accuracy: 0.8493 - val_loss: 0.4764 - val_accuracy: 0.8279
Epoch 9/80
74/74 [==============================] - 14s 184ms/step - loss: 0.3940 - accuracy: 0.8574 - val_loss: 0.5159 - val_accuracy: 0.8211
Epoch 10/80
74/74 [==============================] - 14s 183ms/step - loss: 0.3876 - accuracy: 0.8629 - val_loss: 0.4401 - val_accuracy: 0.8416
Epoch 11/80
74/74 [==============================] - 14s 184ms/step - loss: 0.4075 - accuracy: 0.8459 - val_loss: 0.4741 - val_accuracy: 0.8245
Epoch 12/80
74/74 [==============================] - 14s 184ms/step - loss: 0.4054 - accuracy: 0.8484 - val_loss: 0.4832 - val_accuracy: 0.8313
Epoch 13/80
74/74 [==============================] - 14s 185ms/step - loss: 0.3990 - accuracy: 0.8476 - val_loss: 0.4822 - val_accuracy: 0.8330
Epoch 14/80
74/74 [==============================] - 15s 200ms/step - loss: 0.3897 - accuracy: 0.8595 - val_loss: 0.4300 - val_accuracy: 0.8416
Epoch 15/80
74/74 [==============================] - 16s 219ms/step - loss: 0.4141 - accuracy: 0.8459 - val_loss: 0.4623 - val_accuracy: 0.8450
Epoch 16/80
74/74 [==============================] - 18s 243ms/step - loss: 0.3785 - accuracy: 0.8612 - val_loss: 0.4448 - val_accuracy: 0.8416
Epoch 17/80
74/74 [==============================] - 18s 248ms/step - loss: 0.3796 - accuracy: 0.8578 - val_loss: 0.4371 - val_accuracy: 0.8416
Epoch 18/80
74/74 [==============================] - 20s 269ms/step - loss: 0.3986 - accuracy: 0.8540 - val_loss: 0.4315 - val_accuracy: 0.8484
Epoch 19/80
74/74 [==============================] - 21s 283ms/step - loss: 0.3591 - accuracy: 0.8625 - val_loss: 0.4633 - val_accuracy: 0.8330
Epoch 20/80
74/74 [==============================] - 21s 289ms/step - loss: 0.3971 - accuracy: 0.8421 - val_loss: 0.4394 - val_accuracy: 0.8416
Epoch 21/80
74/74 [==============================] - 23s 306ms/step - loss: 0.3793 - accuracy: 0.8506 - val_loss: 0.4855 - val_accuracy: 0.8160
Epoch 22/80
74/74 [==============================] - 24s 321ms/step - loss: 0.3669 - accuracy: 0.8570 - val_loss: 0.4509 - val_accuracy: 0.8399
Epoch 23/80
74/74 [==============================] - 23s 317ms/step - loss: 0.3808 - accuracy: 0.8557 - val_loss: 0.4186 - val_accuracy: 0.8450
Epoch 24/80
74/74 [==============================] - 23s 305ms/step - loss: 0.3633 - accuracy: 0.8629 - val_loss: 0.4526 - val_accuracy: 0.8365
Epoch 25/80
74/74 [==============================] - 26s 346ms/step - loss: 0.4011 - accuracy: 0.8455 - val_loss: 0.4627 - val_accuracy: 0.8382
Epoch 26/80
74/74 [==============================] - 25s 337ms/step - loss: 0.3569 - accuracy: 0.8668 - val_loss: 0.4531 - val_accuracy: 0.8330
Epoch 27/80
74/74 [==============================] - 25s 344ms/step - loss: 0.3600 - accuracy: 0.8646 - val_loss: 0.4746 - val_accuracy: 0.8348
Epoch 28/80
74/74 [==============================] - 25s 337ms/step - loss: 0.3692 - accuracy: 0.8638 - val_loss: 0.4580 - val_accuracy: 0.8399
Epoch 29/80
74/74 [==============================] - 26s 347ms/step - loss: 0.3644 - accuracy: 0.8638 - val_loss: 0.4454 - val_accuracy: 0.8416
Epoch 30/80
74/74 [==============================] - 26s 352ms/step - loss: 0.3758 - accuracy: 0.8582 - val_loss: 0.4348 - val_accuracy: 0.8467
Epoch 31/80
74/74 [==============================] - 28s 375ms/step - loss: 0.3589 - accuracy: 0.8702 - val_loss: 0.4663 - val_accuracy: 0.8296
Epoch 32/80
74/74 [==============================] - 28s 375ms/step - loss: 0.3601 - accuracy: 0.8608 - val_loss: 0.4550 - val_accuracy: 0.8279
Epoch 33/80
74/74 [==============================] - 28s 382ms/step - loss: 0.3554 - accuracy: 0.8710 - val_loss: 0.4712 - val_accuracy: 0.8313
In [25]:
net.save("models/B3D2DropA2.keras")

Lets now test its performances on the test set:

In [24]:
net.evaluate(test_ds)

23/23 [==============================] - 1s 47ms/step - loss: 0.4168 - accuracy: 0.8474
Out[24]:
[0.4167979061603546, 0.8474114537239075]

The test accuracy is of 84.74%, this is even better than the first model we tested.

Parameters Tuning¶

We are going to try and tune the parameter of the model. Searching for a combination with better performances:

Lower LR decay¶

In [48]:
# Architecture definition LR decay 0.05
inputs = keras.Input((None,None,3))

x = inputs

x = keras.layers.RandomCrop(224, 224)(x)
x = keras.layers.RandomFlip(mode='horizontal')(x)
x = keras.layers.RandomContrast(0.25)(x)

x = keras.layers.Conv2D(64, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.MaxPooling2D(3, strides=3, padding="same")(x)
x = keras.layers.Dropout(0.1)(x)

x = keras.layers.Conv2D(128, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.MaxPooling2D(3, strides=3, padding="same")(x)
x = keras.layers.Dropout(0.1)(x)

x = keras.layers.Conv2D(256, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.Dropout(0.1)(x)

x = keras.layers.GlobalMaxPooling2D()(x)

outputs = keras.layers.Dense(1024, activation="softmax")(x)
x = keras.layers.Dropout(0.2)(x)

outputs = keras.layers.Dense(512, activation="softmax")(x)

outputs = keras.layers.Dense(num_classes, activation="softmax")(x)
net = keras.Model(inputs, outputs)
In [49]:
callback = keras.callbacks.EarlyStopping(monitor='val_loss', patience=10)
In [50]:
lr_schedule = keras.optimizers.schedules.InverseTimeDecay(
    initial_learning_rate = 0.0005,
    decay_steps = 65,
    decay_rate = 0.05)
optimizer = keras.optimizers.Adam(learning_rate=lr_schedule)
net.compile(loss=keras.losses.categorical_crossentropy,
            optimizer=optimizer,
            metrics=['accuracy'])
In [51]:
# Model training
history = net.fit(train_ds_256,
          epochs=80,
          validation_data=val_ds,
            callbacks=[callback])

Epoch 1/80
74/74 [==============================] - 13s 156ms/step - loss: 1.8820 - accuracy: 0.4576 - val_loss: 2.6713 - val_accuracy: 0.3612
Epoch 2/80
74/74 [==============================] - 13s 178ms/step - loss: 1.3386 - accuracy: 0.5547 - val_loss: 1.4557 - val_accuracy: 0.5315
Epoch 3/80
74/74 [==============================] - 14s 184ms/step - loss: 1.1921 - accuracy: 0.5905 - val_loss: 1.2134 - val_accuracy: 0.5963
Epoch 4/80
74/74 [==============================] - 14s 184ms/step - loss: 1.1202 - accuracy: 0.6224 - val_loss: 0.9911 - val_accuracy: 0.6457
Epoch 5/80
74/74 [==============================] - 14s 184ms/step - loss: 1.0416 - accuracy: 0.6424 - val_loss: 0.7881 - val_accuracy: 0.6899
Epoch 6/80
74/74 [==============================] - 14s 185ms/step - loss: 0.9600 - accuracy: 0.6577 - val_loss: 0.7651 - val_accuracy: 0.7087
Epoch 7/80
74/74 [==============================] - 17s 226ms/step - loss: 0.8992 - accuracy: 0.6880 - val_loss: 0.7939 - val_accuracy: 0.7070
Epoch 8/80
74/74 [==============================] - 27s 364ms/step - loss: 0.8580 - accuracy: 0.6918 - val_loss: 0.6931 - val_accuracy: 0.7223
Epoch 9/80
74/74 [==============================] - 32s 441ms/step - loss: 0.8392 - accuracy: 0.6943 - val_loss: 0.6811 - val_accuracy: 0.7513
Epoch 10/80
74/74 [==============================] - 34s 454ms/step - loss: 0.8519 - accuracy: 0.7016 - val_loss: 0.7328 - val_accuracy: 0.7274
Epoch 11/80
74/74 [==============================] - 34s 455ms/step - loss: 0.7715 - accuracy: 0.7233 - val_loss: 0.7701 - val_accuracy: 0.7155
Epoch 12/80
74/74 [==============================] - 34s 455ms/step - loss: 0.7909 - accuracy: 0.7190 - val_loss: 0.6620 - val_accuracy: 0.7496
Epoch 13/80
74/74 [==============================] - 34s 456ms/step - loss: 0.7477 - accuracy: 0.7331 - val_loss: 0.6665 - val_accuracy: 0.7598
Epoch 14/80
74/74 [==============================] - 34s 456ms/step - loss: 0.7420 - accuracy: 0.7212 - val_loss: 0.6552 - val_accuracy: 0.7428
Epoch 15/80
74/74 [==============================] - 34s 456ms/step - loss: 0.7379 - accuracy: 0.7263 - val_loss: 0.7248 - val_accuracy: 0.7206
Epoch 16/80
74/74 [==============================] - 34s 457ms/step - loss: 0.6935 - accuracy: 0.7446 - val_loss: 0.6758 - val_accuracy: 0.7581
Epoch 17/80
74/74 [==============================] - 34s 457ms/step - loss: 0.7128 - accuracy: 0.7458 - val_loss: 0.6316 - val_accuracy: 0.7547
Epoch 18/80
74/74 [==============================] - 34s 458ms/step - loss: 0.6565 - accuracy: 0.7625 - val_loss: 0.6084 - val_accuracy: 0.7802
Epoch 19/80
74/74 [==============================] - 34s 459ms/step - loss: 0.6281 - accuracy: 0.7735 - val_loss: 0.5692 - val_accuracy: 0.7939
Epoch 20/80
74/74 [==============================] - 34s 458ms/step - loss: 0.6568 - accuracy: 0.7688 - val_loss: 0.6026 - val_accuracy: 0.7802
Epoch 21/80
74/74 [==============================] - 34s 460ms/step - loss: 0.6298 - accuracy: 0.7642 - val_loss: 0.6680 - val_accuracy: 0.7666
Epoch 22/80
74/74 [==============================] - 34s 459ms/step - loss: 0.6002 - accuracy: 0.7739 - val_loss: 0.5836 - val_accuracy: 0.7922
Epoch 23/80
74/74 [==============================] - 34s 461ms/step - loss: 0.6209 - accuracy: 0.7718 - val_loss: 0.6036 - val_accuracy: 0.7632
Epoch 24/80
74/74 [==============================] - 34s 460ms/step - loss: 0.5795 - accuracy: 0.7791 - val_loss: 0.5369 - val_accuracy: 0.8160
Epoch 25/80
74/74 [==============================] - 34s 460ms/step - loss: 0.5688 - accuracy: 0.7871 - val_loss: 0.6609 - val_accuracy: 0.7700
Epoch 26/80
74/74 [==============================] - 34s 460ms/step - loss: 0.5724 - accuracy: 0.7859 - val_loss: 0.6368 - val_accuracy: 0.7717
Epoch 27/80
74/74 [==============================] - 34s 462ms/step - loss: 0.5841 - accuracy: 0.7867 - val_loss: 0.6143 - val_accuracy: 0.7888
Epoch 28/80
74/74 [==============================] - 34s 461ms/step - loss: 0.5593 - accuracy: 0.7944 - val_loss: 0.6207 - val_accuracy: 0.7683
Epoch 29/80
74/74 [==============================] - 34s 462ms/step - loss: 0.5478 - accuracy: 0.8080 - val_loss: 0.5421 - val_accuracy: 0.8143
Epoch 30/80
74/74 [==============================] - 35s 463ms/step - loss: 0.5553 - accuracy: 0.7957 - val_loss: 0.6288 - val_accuracy: 0.7785
Epoch 31/80
74/74 [==============================] - 35s 463ms/step - loss: 0.5364 - accuracy: 0.8033 - val_loss: 0.5677 - val_accuracy: 0.7956
Epoch 32/80
74/74 [==============================] - 35s 462ms/step - loss: 0.5353 - accuracy: 0.8008 - val_loss: 0.5050 - val_accuracy: 0.8177
Epoch 33/80
74/74 [==============================] - 35s 463ms/step - loss: 0.5327 - accuracy: 0.7931 - val_loss: 0.5286 - val_accuracy: 0.8092
Epoch 34/80
74/74 [==============================] - 35s 464ms/step - loss: 0.4982 - accuracy: 0.8182 - val_loss: 0.5487 - val_accuracy: 0.7973
Epoch 35/80
74/74 [==============================] - 35s 465ms/step - loss: 0.5025 - accuracy: 0.8037 - val_loss: 0.5164 - val_accuracy: 0.8143
Epoch 36/80
74/74 [==============================] - 35s 463ms/step - loss: 0.4868 - accuracy: 0.8238 - val_loss: 0.5123 - val_accuracy: 0.8143
Epoch 37/80
74/74 [==============================] - 35s 465ms/step - loss: 0.4997 - accuracy: 0.8072 - val_loss: 0.4611 - val_accuracy: 0.8348
Epoch 38/80
74/74 [==============================] - 35s 463ms/step - loss: 0.4617 - accuracy: 0.8263 - val_loss: 0.5251 - val_accuracy: 0.8109
Epoch 39/80
74/74 [==============================] - 35s 465ms/step - loss: 0.4830 - accuracy: 0.8169 - val_loss: 0.5330 - val_accuracy: 0.8041
Epoch 40/80
74/74 [==============================] - 35s 466ms/step - loss: 0.4690 - accuracy: 0.8229 - val_loss: 0.5794 - val_accuracy: 0.7700
Epoch 41/80
74/74 [==============================] - 35s 465ms/step - loss: 0.4657 - accuracy: 0.8250 - val_loss: 0.4679 - val_accuracy: 0.8382
Epoch 42/80
74/74 [==============================] - 35s 463ms/step - loss: 0.4941 - accuracy: 0.8123 - val_loss: 0.4858 - val_accuracy: 0.8228
Epoch 43/80
74/74 [==============================] - 35s 466ms/step - loss: 0.4500 - accuracy: 0.8297 - val_loss: 0.4614 - val_accuracy: 0.8365
Epoch 44/80
74/74 [==============================] - 35s 464ms/step - loss: 0.4776 - accuracy: 0.8169 - val_loss: 0.5513 - val_accuracy: 0.7853
Epoch 45/80
74/74 [==============================] - 35s 466ms/step - loss: 0.4634 - accuracy: 0.8280 - val_loss: 0.5430 - val_accuracy: 0.7990
Epoch 46/80
74/74 [==============================] - 35s 464ms/step - loss: 0.4478 - accuracy: 0.8365 - val_loss: 0.5213 - val_accuracy: 0.8177
Epoch 47/80
74/74 [==============================] - 35s 465ms/step - loss: 0.4520 - accuracy: 0.8365 - val_loss: 0.4982 - val_accuracy: 0.8177
In [52]:
# Visualization of the learning curves

plt.subplot(1, 2, 1)
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.ylim([0.0, 2.0])
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.legend(['train', 'valid'])

plt.subplot(1, 2, 2)
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.ylim([0.5, 1.0])
plt.xlabel('Epoch')
plt.ylabel('Accuracy')
plt.legend(['train', 'valid'])
Out[52]:
<matplotlib.legend.Legend at 0x257e022c5b0>
In [53]:
net.save("models/B3D2DropAFine1.keras")

Different kernel size (5)¶

In [46]:
# Architecture definition 5
inputs = keras.Input((None,None,3))

x = inputs

x = keras.layers.RandomCrop(224, 224)(x)
x = keras.layers.RandomFlip(mode='horizontal')(x)
x = keras.layers.RandomContrast(0.25)(x)

x = keras.layers.Conv2D(64, 5, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.MaxPooling2D(3, strides=3, padding="same")(x)
x = keras.layers.Dropout(0.1)(x)

x = keras.layers.Conv2D(128, 5, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.MaxPooling2D(3, strides=3, padding="same")(x)
x = keras.layers.Dropout(0.1)(x)

x = keras.layers.Conv2D(256, 5, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.Dropout(0.1)(x)

x = keras.layers.GlobalMaxPooling2D()(x)

outputs = keras.layers.Dense(1024, activation="softmax")(x)
x = keras.layers.Dropout(0.2)(x)

outputs = keras.layers.Dense(512, activation="softmax")(x)

outputs = keras.layers.Dense(num_classes, activation="softmax")(x)
net = keras.Model(inputs, outputs)
In [47]:
callback = keras.callbacks.EarlyStopping(monitor='val_loss', patience=10)
In [34]:
lr_schedule = keras.optimizers.schedules.InverseTimeDecay(
    initial_learning_rate = 0.0005,
    decay_steps = 65,
    decay_rate = 0.08)
optimizer = keras.optimizers.Adam(learning_rate=lr_schedule)
net.compile(loss=keras.losses.categorical_crossentropy,
            optimizer=optimizer,
            metrics=['accuracy'])
In [35]:
# Model training
history = net.fit(train_ds_256,
          epochs=80,
          validation_data=val_ds,
            callbacks=[callback])

Epoch 1/80
74/74 [==============================] - 70s 628ms/step - loss: 1.9723 - accuracy: 0.4572 - val_loss: 3.0639 - val_accuracy: 0.4123
Epoch 2/80
74/74 [==============================] - 34s 467ms/step - loss: 1.2672 - accuracy: 0.5568 - val_loss: 1.9430 - val_accuracy: 0.4514
Epoch 3/80
74/74 [==============================] - 36s 485ms/step - loss: 1.1827 - accuracy: 0.5892 - val_loss: 0.9465 - val_accuracy: 0.6320
Epoch 4/80
74/74 [==============================] - 35s 480ms/step - loss: 1.0537 - accuracy: 0.6377 - val_loss: 0.9106 - val_accuracy: 0.6491
Epoch 5/80
74/74 [==============================] - 36s 486ms/step - loss: 1.0364 - accuracy: 0.6394 - val_loss: 0.7594 - val_accuracy: 0.7206
Epoch 6/80
74/74 [==============================] - 35s 479ms/step - loss: 0.9545 - accuracy: 0.6671 - val_loss: 0.7358 - val_accuracy: 0.7138
Epoch 7/80
74/74 [==============================] - 36s 483ms/step - loss: 0.8745 - accuracy: 0.6854 - val_loss: 0.7895 - val_accuracy: 0.7121
Epoch 8/80
74/74 [==============================] - 36s 486ms/step - loss: 0.8328 - accuracy: 0.7080 - val_loss: 0.7764 - val_accuracy: 0.7070
Epoch 9/80
74/74 [==============================] - 36s 481ms/step - loss: 0.8246 - accuracy: 0.7050 - val_loss: 0.6681 - val_accuracy: 0.7308
Epoch 10/80
74/74 [==============================] - 36s 487ms/step - loss: 0.7454 - accuracy: 0.7271 - val_loss: 0.7120 - val_accuracy: 0.7342
Epoch 11/80
74/74 [==============================] - 36s 486ms/step - loss: 0.7493 - accuracy: 0.7186 - val_loss: 0.6116 - val_accuracy: 0.7666
Epoch 12/80
74/74 [==============================] - 36s 483ms/step - loss: 0.7184 - accuracy: 0.7382 - val_loss: 0.6680 - val_accuracy: 0.7615
Epoch 13/80
74/74 [==============================] - 36s 485ms/step - loss: 0.7030 - accuracy: 0.7510 - val_loss: 0.6577 - val_accuracy: 0.7666
Epoch 14/80
74/74 [==============================] - 36s 485ms/step - loss: 0.6629 - accuracy: 0.7569 - val_loss: 0.6102 - val_accuracy: 0.7683
Epoch 15/80
74/74 [==============================] - 36s 491ms/step - loss: 0.6383 - accuracy: 0.7629 - val_loss: 0.7078 - val_accuracy: 0.7564
Epoch 16/80
74/74 [==============================] - 36s 491ms/step - loss: 0.6192 - accuracy: 0.7722 - val_loss: 0.6290 - val_accuracy: 0.7700
Epoch 17/80
74/74 [==============================] - 36s 485ms/step - loss: 0.5985 - accuracy: 0.7825 - val_loss: 0.6485 - val_accuracy: 0.7462
Epoch 18/80
74/74 [==============================] - 36s 488ms/step - loss: 0.6086 - accuracy: 0.7705 - val_loss: 0.8143 - val_accuracy: 0.7223
Epoch 19/80
74/74 [==============================] - 36s 491ms/step - loss: 0.5763 - accuracy: 0.7808 - val_loss: 0.6018 - val_accuracy: 0.7836
Epoch 20/80
74/74 [==============================] - 36s 489ms/step - loss: 0.5878 - accuracy: 0.7850 - val_loss: 0.5289 - val_accuracy: 0.8262
Epoch 21/80
74/74 [==============================] - 36s 492ms/step - loss: 0.5544 - accuracy: 0.7969 - val_loss: 0.5722 - val_accuracy: 0.8041
Epoch 22/80
74/74 [==============================] - 36s 491ms/step - loss: 0.5402 - accuracy: 0.8020 - val_loss: 0.6061 - val_accuracy: 0.7836
Epoch 23/80
74/74 [==============================] - 36s 491ms/step - loss: 0.5272 - accuracy: 0.8016 - val_loss: 0.7183 - val_accuracy: 0.7513
Epoch 24/80
74/74 [==============================] - 36s 491ms/step - loss: 0.5300 - accuracy: 0.8059 - val_loss: 0.6860 - val_accuracy: 0.7428
Epoch 25/80
74/74 [==============================] - 36s 491ms/step - loss: 0.4716 - accuracy: 0.8242 - val_loss: 0.5214 - val_accuracy: 0.8143
Epoch 26/80
74/74 [==============================] - 36s 490ms/step - loss: 0.4892 - accuracy: 0.8203 - val_loss: 0.5150 - val_accuracy: 0.8194
Epoch 27/80
74/74 [==============================] - 36s 487ms/step - loss: 0.4843 - accuracy: 0.8267 - val_loss: 0.6179 - val_accuracy: 0.7717
Epoch 28/80
74/74 [==============================] - 36s 487ms/step - loss: 0.4709 - accuracy: 0.8233 - val_loss: 0.5313 - val_accuracy: 0.8109
Epoch 29/80
74/74 [==============================] - 36s 485ms/step - loss: 0.4921 - accuracy: 0.8152 - val_loss: 0.6142 - val_accuracy: 0.7802
Epoch 30/80
74/74 [==============================] - 36s 486ms/step - loss: 0.4661 - accuracy: 0.8314 - val_loss: 0.6157 - val_accuracy: 0.7683
Epoch 31/80
74/74 [==============================] - 36s 490ms/step - loss: 0.4270 - accuracy: 0.8382 - val_loss: 0.5151 - val_accuracy: 0.8126
Epoch 32/80
74/74 [==============================] - 36s 492ms/step - loss: 0.4274 - accuracy: 0.8408 - val_loss: 0.5333 - val_accuracy: 0.8211
Epoch 33/80
74/74 [==============================] - 37s 494ms/step - loss: 0.4425 - accuracy: 0.8327 - val_loss: 0.5284 - val_accuracy: 0.8160
Epoch 34/80
74/74 [==============================] - 37s 495ms/step - loss: 0.4163 - accuracy: 0.8433 - val_loss: 0.5098 - val_accuracy: 0.8126
Epoch 35/80
74/74 [==============================] - 37s 495ms/step - loss: 0.4106 - accuracy: 0.8510 - val_loss: 0.5483 - val_accuracy: 0.7956
Epoch 36/80
74/74 [==============================] - 37s 496ms/step - loss: 0.4352 - accuracy: 0.8382 - val_loss: 0.5015 - val_accuracy: 0.8313
Epoch 37/80
74/74 [==============================] - 37s 497ms/step - loss: 0.4004 - accuracy: 0.8540 - val_loss: 0.4969 - val_accuracy: 0.8330
Epoch 38/80
74/74 [==============================] - 37s 497ms/step - loss: 0.4063 - accuracy: 0.8472 - val_loss: 0.5872 - val_accuracy: 0.7785
Epoch 39/80
74/74 [==============================] - 37s 498ms/step - loss: 0.3842 - accuracy: 0.8561 - val_loss: 0.4957 - val_accuracy: 0.8296
Epoch 40/80
74/74 [==============================] - 37s 500ms/step - loss: 0.3688 - accuracy: 0.8655 - val_loss: 0.5243 - val_accuracy: 0.8143
Epoch 41/80
74/74 [==============================] - 37s 499ms/step - loss: 0.3777 - accuracy: 0.8527 - val_loss: 0.4983 - val_accuracy: 0.8262
Epoch 42/80
74/74 [==============================] - 37s 501ms/step - loss: 0.3728 - accuracy: 0.8557 - val_loss: 0.4954 - val_accuracy: 0.8262
Epoch 43/80
74/74 [==============================] - 37s 500ms/step - loss: 0.3836 - accuracy: 0.8599 - val_loss: 0.5006 - val_accuracy: 0.8075
Epoch 44/80
74/74 [==============================] - 37s 501ms/step - loss: 0.3626 - accuracy: 0.8685 - val_loss: 0.4967 - val_accuracy: 0.8296
Epoch 45/80
74/74 [==============================] - 37s 501ms/step - loss: 0.3672 - accuracy: 0.8659 - val_loss: 0.4865 - val_accuracy: 0.8228
Epoch 46/80
74/74 [==============================] - 37s 501ms/step - loss: 0.3436 - accuracy: 0.8748 - val_loss: 0.4235 - val_accuracy: 0.8535
Epoch 47/80
74/74 [==============================] - 37s 501ms/step - loss: 0.3676 - accuracy: 0.8612 - val_loss: 0.4728 - val_accuracy: 0.8330
Epoch 48/80
74/74 [==============================] - 38s 504ms/step - loss: 0.3482 - accuracy: 0.8655 - val_loss: 0.4464 - val_accuracy: 0.8399
Epoch 49/80
74/74 [==============================] - 38s 502ms/step - loss: 0.3303 - accuracy: 0.8770 - val_loss: 0.4197 - val_accuracy: 0.8518
Epoch 50/80
74/74 [==============================] - 38s 503ms/step - loss: 0.3237 - accuracy: 0.8782 - val_loss: 0.4473 - val_accuracy: 0.8484
Epoch 51/80
74/74 [==============================] - 38s 503ms/step - loss: 0.3364 - accuracy: 0.8765 - val_loss: 0.4770 - val_accuracy: 0.8296
Epoch 52/80
74/74 [==============================] - 38s 503ms/step - loss: 0.3416 - accuracy: 0.8663 - val_loss: 0.4995 - val_accuracy: 0.8143
Epoch 53/80
74/74 [==============================] - 38s 504ms/step - loss: 0.3153 - accuracy: 0.8804 - val_loss: 0.4546 - val_accuracy: 0.8433
Epoch 54/80
74/74 [==============================] - 38s 504ms/step - loss: 0.3368 - accuracy: 0.8757 - val_loss: 0.4601 - val_accuracy: 0.8450
Epoch 55/80
74/74 [==============================] - 38s 505ms/step - loss: 0.3104 - accuracy: 0.8817 - val_loss: 0.5010 - val_accuracy: 0.8279
Epoch 56/80
74/74 [==============================] - 38s 504ms/step - loss: 0.3179 - accuracy: 0.8761 - val_loss: 0.4905 - val_accuracy: 0.8348
Epoch 57/80
74/74 [==============================] - 38s 504ms/step - loss: 0.3270 - accuracy: 0.8770 - val_loss: 0.4539 - val_accuracy: 0.8348
Epoch 58/80
74/74 [==============================] - 38s 507ms/step - loss: 0.2982 - accuracy: 0.8876 - val_loss: 0.5045 - val_accuracy: 0.8245
Epoch 59/80
74/74 [==============================] - 38s 506ms/step - loss: 0.3111 - accuracy: 0.8804 - val_loss: 0.4730 - val_accuracy: 0.8416
In [36]:
# Visualization of the learning curves

plt.subplot(1, 2, 1)
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.ylim([0.0, 2.0])
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.legend(['train', 'valid'])

plt.subplot(1, 2, 2)
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.ylim([0.5, 1.0])
plt.xlabel('Epoch')
plt.ylabel('Accuracy')
plt.legend(['train', 'valid'])
Out[36]:
<matplotlib.legend.Legend at 0x2580fd37ca0>
In [37]:
net.save("models/B3D2DropAFine2.keras")

Back to the RMSprop optimizer¶

In [38]:
# Architecture definition
inputs = keras.Input((None,None,3))

x = inputs

x = keras.layers.RandomCrop(224, 224)(x)
x = keras.layers.RandomFlip(mode='horizontal')(x)
x = keras.layers.RandomContrast(0.25)(x)

x = keras.layers.Conv2D(64, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.MaxPooling2D(3, strides=3, padding="same")(x)
x = keras.layers.Dropout(0.1)(x)

x = keras.layers.Conv2D(128, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.MaxPooling2D(3, strides=3, padding="same")(x)
x = keras.layers.Dropout(0.1)(x)

x = keras.layers.Conv2D(256, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.Dropout(0.1)(x)

x = keras.layers.GlobalMaxPooling2D()(x)

outputs = keras.layers.Dense(1024, activation="softmax")(x)
x = keras.layers.Dropout(0.2)(x)

outputs = keras.layers.Dense(512, activation="softmax")(x)

outputs = keras.layers.Dense(num_classes, activation="softmax")(x)
net = keras.Model(inputs, outputs)
In [39]:
callback = keras.callbacks.EarlyStopping(monitor='val_loss', patience=10)
In [40]:
lr_schedule = keras.optimizers.schedules.InverseTimeDecay(
    initial_learning_rate = 0.0005,
    decay_steps = 65,
    decay_rate = 0.08)
optimizer = keras.optimizers.RMSprop(learning_rate=lr_schedule)
net.compile(loss=keras.losses.categorical_crossentropy,
            optimizer=optimizer,
            metrics=['accuracy'])
In [41]:
# Model training
history = net.fit(train_ds_256,
          epochs=80,
          validation_data=val_ds,
            callbacks=[callback])

Epoch 1/80
74/74 [==============================] - 38s 469ms/step - loss: 2.0206 - accuracy: 0.4534 - val_loss: 2.5109 - val_accuracy: 0.4310
Epoch 2/80
74/74 [==============================] - 35s 464ms/step - loss: 1.4229 - accuracy: 0.5568 - val_loss: 1.1077 - val_accuracy: 0.6031
Epoch 3/80
74/74 [==============================] - 35s 464ms/step - loss: 1.2648 - accuracy: 0.5956 - val_loss: 1.3544 - val_accuracy: 0.5826
Epoch 4/80
74/74 [==============================] - 35s 463ms/step - loss: 1.0763 - accuracy: 0.6415 - val_loss: 0.8185 - val_accuracy: 0.6899
Epoch 5/80
74/74 [==============================] - 35s 463ms/step - loss: 1.0171 - accuracy: 0.6607 - val_loss: 0.9566 - val_accuracy: 0.6644
Epoch 6/80
74/74 [==============================] - 35s 464ms/step - loss: 0.9734 - accuracy: 0.6611 - val_loss: 1.0071 - val_accuracy: 0.6644
Epoch 7/80
74/74 [==============================] - 35s 463ms/step - loss: 0.9270 - accuracy: 0.6858 - val_loss: 0.7377 - val_accuracy: 0.7036
Epoch 8/80
74/74 [==============================] - 35s 464ms/step - loss: 0.8417 - accuracy: 0.7033 - val_loss: 0.6614 - val_accuracy: 0.7445
Epoch 9/80
74/74 [==============================] - 34s 462ms/step - loss: 0.8066 - accuracy: 0.7114 - val_loss: 0.7229 - val_accuracy: 0.7291
Epoch 10/80
74/74 [==============================] - 34s 463ms/step - loss: 0.8105 - accuracy: 0.7114 - val_loss: 0.8800 - val_accuracy: 0.6934
Epoch 11/80
74/74 [==============================] - 34s 462ms/step - loss: 0.7526 - accuracy: 0.7305 - val_loss: 0.7667 - val_accuracy: 0.7274
Epoch 12/80
74/74 [==============================] - 35s 463ms/step - loss: 0.7365 - accuracy: 0.7437 - val_loss: 0.6912 - val_accuracy: 0.7445
Epoch 13/80
74/74 [==============================] - 35s 463ms/step - loss: 0.7325 - accuracy: 0.7335 - val_loss: 0.5892 - val_accuracy: 0.7871
Epoch 14/80
74/74 [==============================] - 34s 462ms/step - loss: 0.7110 - accuracy: 0.7441 - val_loss: 0.7786 - val_accuracy: 0.7036
Epoch 15/80
74/74 [==============================] - 34s 463ms/step - loss: 0.6913 - accuracy: 0.7476 - val_loss: 0.7168 - val_accuracy: 0.7462
Epoch 16/80
74/74 [==============================] - 34s 462ms/step - loss: 0.6759 - accuracy: 0.7565 - val_loss: 0.6973 - val_accuracy: 0.7479
Epoch 17/80
74/74 [==============================] - 35s 467ms/step - loss: 0.6650 - accuracy: 0.7612 - val_loss: 0.6009 - val_accuracy: 0.7888
Epoch 18/80
74/74 [==============================] - 35s 466ms/step - loss: 0.6584 - accuracy: 0.7607 - val_loss: 0.7154 - val_accuracy: 0.7598
Epoch 19/80
74/74 [==============================] - 35s 463ms/step - loss: 0.6326 - accuracy: 0.7782 - val_loss: 0.6790 - val_accuracy: 0.7751
Epoch 20/80
74/74 [==============================] - 35s 463ms/step - loss: 0.6059 - accuracy: 0.7786 - val_loss: 0.5421 - val_accuracy: 0.8041
Epoch 21/80
74/74 [==============================] - 35s 464ms/step - loss: 0.6082 - accuracy: 0.7795 - val_loss: 0.5861 - val_accuracy: 0.7785
Epoch 22/80
74/74 [==============================] - 35s 463ms/step - loss: 0.6017 - accuracy: 0.7769 - val_loss: 0.4950 - val_accuracy: 0.8126
Epoch 23/80
74/74 [==============================] - 35s 463ms/step - loss: 0.5798 - accuracy: 0.7867 - val_loss: 0.5994 - val_accuracy: 0.7666
Epoch 24/80
74/74 [==============================] - 35s 464ms/step - loss: 0.5846 - accuracy: 0.7820 - val_loss: 0.6202 - val_accuracy: 0.7666
Epoch 25/80
74/74 [==============================] - 35s 464ms/step - loss: 0.5749 - accuracy: 0.7859 - val_loss: 0.5482 - val_accuracy: 0.7836
Epoch 26/80
74/74 [==============================] - 35s 464ms/step - loss: 0.5405 - accuracy: 0.8037 - val_loss: 0.5649 - val_accuracy: 0.7836
Epoch 27/80
74/74 [==============================] - 35s 465ms/step - loss: 0.5723 - accuracy: 0.7961 - val_loss: 0.5683 - val_accuracy: 0.7905
Epoch 28/80
74/74 [==============================] - 35s 465ms/step - loss: 0.5322 - accuracy: 0.8118 - val_loss: 0.5604 - val_accuracy: 0.7836
Epoch 29/80
74/74 [==============================] - 35s 465ms/step - loss: 0.5373 - accuracy: 0.8054 - val_loss: 0.5751 - val_accuracy: 0.7973
Epoch 30/80
74/74 [==============================] - 35s 468ms/step - loss: 0.5539 - accuracy: 0.7999 - val_loss: 0.5139 - val_accuracy: 0.8092
Epoch 31/80
74/74 [==============================] - 35s 472ms/step - loss: 0.5313 - accuracy: 0.7978 - val_loss: 0.5690 - val_accuracy: 0.7853
Epoch 32/80
74/74 [==============================] - 35s 470ms/step - loss: 0.5071 - accuracy: 0.8110 - val_loss: 0.5958 - val_accuracy: 0.7871
In [44]:
# Visualization of the learning curves

plt.subplot(1, 2, 1)
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.ylim([0.0, 2.0])
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.legend(['train', 'valid'])

plt.subplot(1, 2, 2)
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.ylim([0.5, 1.0])
plt.xlabel('Epoch')
plt.ylabel('Accuracy')
plt.legend(['train', 'valid'])
Out[44]:
<matplotlib.legend.Legend at 0x257800d99a0>
In [43]:
net.save("models/B3D2DropAFine3.keras")

None of the modification resulted in better performances of the model.

Segmented Data¶

Most of the literature on flower recognition with CNN (Y Liu, 2016, I Gogul, 2017, T Nguyen) cites background as one of the biggest problem in flower classification. This is because flowers, because of their environment, tend to have complex background, and that complexity can hurt the performances of a classification model.
Even if literature cites different segmentation methods to resolve the issue, we are going to utilize a rather "simple" (for what regards application) method: Segment Anything a Kirillov, 2023. More specifically, we are going to use LANG segment anything, an open-source project that combines the power of instance segmentation and text prompts to generate masks for specific objects in images. In this way, segmenting our images will be as easy as running the model specifying a prompt like "flower".

In [ ]:
# Download the model
%pip install -U git+https://github.com/luca-medeiros/lang-segment-anything.git

Collecting git+https://github.com/luca-medeiros/lang-segment-anything.git
  Cloning https://github.com/luca-medeiros/lang-segment-anything.git to /tmp/pip-req-build-pxs8esfi
  Running command git clone --filter=blob:none --quiet https://github.com/luca-medeiros/lang-segment-anything.git /tmp/pip-req-build-pxs8esfi
  Resolved https://github.com/luca-medeiros/lang-segment-anything.git to commit f4ca2960a97cb73aff2c459a9d0f06c7674452e4
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting groundingdino@ git+https://github.com/IDEA-Research/GroundingDINO.git (from lang-sam==0.1.0)
  Cloning https://github.com/IDEA-Research/GroundingDINO.git to /tmp/pip-install-2nsrj_78/groundingdino_0e63f605efad4244af80591a5f5bfad1
  Running command git clone --filter=blob:none --quiet https://github.com/IDEA-Research/GroundingDINO.git /tmp/pip-install-2nsrj_78/groundingdino_0e63f605efad4244af80591a5f5bfad1
  Resolved https://github.com/IDEA-Research/GroundingDINO.git to commit c023468fafeb0933d06aa8ba49d96089ce9a6af3
  Preparing metadata (setup.py) ... done
Collecting segment-anything@ git+https://github.com/facebookresearch/segment-anything.git (from lang-sam==0.1.0)
  Cloning https://github.com/facebookresearch/segment-anything.git to /tmp/pip-install-2nsrj_78/segment-anything_e44e8a45738a46b0a62d55f146b89e8b
  Running command git clone --filter=blob:none --quiet https://github.com/facebookresearch/segment-anything.git /tmp/pip-install-2nsrj_78/segment-anything_e44e8a45738a46b0a62d55f146b89e8b
  Resolved https://github.com/facebookresearch/segment-anything.git to commit 6fdee8f2727f4506cfbbe553e23b895e27956588
  Preparing metadata (setup.py) ... done
Collecting Pillow==9.3.0 (from lang-sam==0.1.0)
  Downloading Pillow-9.3.0-cp310-cp310-manylinux_2_28_x86_64.whl (3.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.3/3.3 MB 32.8 MB/s eta 0:00:00
Collecting gradio<4.0.0,>=3.24.1 (from lang-sam==0.1.0)
  Downloading gradio-3.50.2-py3-none-any.whl (20.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 20.3/20.3 MB 65.2 MB/s eta 0:00:00
Collecting huggingface-hub<0.17.0,>=0.16.4 (from lang-sam==0.1.0)
  Downloading huggingface_hub-0.16.4-py3-none-any.whl (268 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 268.8/268.8 kB 37.1 MB/s eta 0:00:00
Collecting lightning<3.0.0,>=2.0.1 (from lang-sam==0.1.0)
  Downloading lightning-2.2.1-py3-none-any.whl (2.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 66.0 MB/s eta 0:00:00
Requirement already satisfied: numpy<2.0.0,>=1.24.2 in /usr/local/lib/python3.10/dist-packages (from lang-sam==0.1.0) (1.25.2)
Requirement already satisfied: opencv_python<5.0.0.0,>=4.7.0.72 in /usr/local/lib/python3.10/dist-packages (from lang-sam==0.1.0) (4.8.0.76)
Requirement already satisfied: transformers<5.0.0,>=4.27.4 in /usr/local/lib/python3.10/dist-packages (from lang-sam==0.1.0) (4.38.2)
Collecting aiofiles<24.0,>=22.0 (from gradio<4.0.0,>=3.24.1->lang-sam==0.1.0)
  Downloading aiofiles-23.2.1-py3-none-any.whl (15 kB)
Requirement already satisfied: altair<6.0,>=4.2.0 in /usr/local/lib/python3.10/dist-packages (from gradio<4.0.0,>=3.24.1->lang-sam==0.1.0) (4.2.2)
Collecting fastapi (from gradio<4.0.0,>=3.24.1->lang-sam==0.1.0)
  Downloading fastapi-0.110.0-py3-none-any.whl (92 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 92.1/92.1 kB 15.3 MB/s eta 0:00:00
Collecting ffmpy (from gradio<4.0.0,>=3.24.1->lang-sam==0.1.0)
  Downloading ffmpy-0.3.2.tar.gz (5.5 kB)
  Preparing metadata (setup.py) ... done
Collecting gradio-client==0.6.1 (from gradio<4.0.0,>=3.24.1->lang-sam==0.1.0)
  Downloading gradio_client-0.6.1-py3-none-any.whl (299 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 299.2/299.2 kB 36.9 MB/s eta 0:00:00
Collecting httpx (from gradio<4.0.0,>=3.24.1->lang-sam==0.1.0)
  Downloading httpx-0.27.0-py3-none-any.whl (75 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 75.6/75.6 kB 13.6 MB/s eta 0:00:00
Requirement already satisfied: importlib-resources<7.0,>=1.3 in /usr/local/lib/python3.10/dist-packages (from gradio<4.0.0,>=3.24.1->lang-sam==0.1.0) (6.3.0)
Requirement already satisfied: jinja2<4.0 in /usr/local/lib/python3.10/dist-packages (from gradio<4.0.0,>=3.24.1->lang-sam==0.1.0) (3.1.3)
Requirement already satisfied: markupsafe~=2.0 in /usr/local/lib/python3.10/dist-packages (from gradio<4.0.0,>=3.24.1->lang-sam==0.1.0) (2.1.5)
Requirement already satisfied: matplotlib~=3.0 in /usr/local/lib/python3.10/dist-packages (from gradio<4.0.0,>=3.24.1->lang-sam==0.1.0) (3.7.1)
Collecting orjson~=3.0 (from gradio<4.0.0,>=3.24.1->lang-sam==0.1.0)
  Downloading orjson-3.9.15-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 138.5/138.5 kB 21.4 MB/s eta 0:00:00
Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from gradio<4.0.0,>=3.24.1->lang-sam==0.1.0) (24.0)
Requirement already satisfied: pandas<3.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from gradio<4.0.0,>=3.24.1->lang-sam==0.1.0) (1.5.3)
Requirement already satisfied: pydantic!=1.8,!=1.8.1,!=2.0.0,!=2.0.1,<3.0.0,>=1.7.4 in /usr/local/lib/python3.10/dist-packages (from gradio<4.0.0,>=3.24.1->lang-sam==0.1.0) (2.6.4)
Collecting pydub (from gradio<4.0.0,>=3.24.1->lang-sam==0.1.0)
  Downloading pydub-0.25.1-py2.py3-none-any.whl (32 kB)
Collecting python-multipart (from gradio<4.0.0,>=3.24.1->lang-sam==0.1.0)
  Downloading python_multipart-0.0.9-py3-none-any.whl (22 kB)
Requirement already satisfied: pyyaml<7.0,>=5.0 in /usr/local/lib/python3.10/dist-packages (from gradio<4.0.0,>=3.24.1->lang-sam==0.1.0) (6.0.1)
Requirement already satisfied: requests~=2.0 in /usr/local/lib/python3.10/dist-packages (from gradio<4.0.0,>=3.24.1->lang-sam==0.1.0) (2.31.0)
Collecting semantic-version~=2.0 (from gradio<4.0.0,>=3.24.1->lang-sam==0.1.0)
  Downloading semantic_version-2.10.0-py2.py3-none-any.whl (15 kB)
Requirement already satisfied: typing-extensions~=4.0 in /usr/local/lib/python3.10/dist-packages (from gradio<4.0.0,>=3.24.1->lang-sam==0.1.0) (4.10.0)
Collecting uvicorn>=0.14.0 (from gradio<4.0.0,>=3.24.1->lang-sam==0.1.0)
  Downloading uvicorn-0.29.0-py3-none-any.whl (60 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 60.8/60.8 kB 10.6 MB/s eta 0:00:00
Collecting websockets<12.0,>=10.0 (from gradio<4.0.0,>=3.24.1->lang-sam==0.1.0)
  Downloading websockets-11.0.3-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (129 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 129.9/129.9 kB 17.5 MB/s eta 0:00:00
Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from gradio-client==0.6.1->gradio<4.0.0,>=3.24.1->lang-sam==0.1.0) (2023.6.0)
Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<0.17.0,>=0.16.4->lang-sam==0.1.0) (3.13.1)
Requirement already satisfied: tqdm>=4.42.1 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<0.17.0,>=0.16.4->lang-sam==0.1.0) (4.66.2)
Collecting lightning-utilities<2.0,>=0.8.0 (from lightning<3.0.0,>=2.0.1->lang-sam==0.1.0)
  Downloading lightning_utilities-0.11.0-py3-none-any.whl (25 kB)
Requirement already satisfied: torch<4.0,>=1.13.0 in /usr/local/lib/python3.10/dist-packages (from lightning<3.0.0,>=2.0.1->lang-sam==0.1.0) (2.2.1+cu121)
Collecting torchmetrics<3.0,>=0.7.0 (from lightning<3.0.0,>=2.0.1->lang-sam==0.1.0)
  Downloading torchmetrics-1.3.2-py3-none-any.whl (841 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 841.5/841.5 kB 54.0 MB/s eta 0:00:00
Collecting pytorch-lightning (from lightning<3.0.0,>=2.0.1->lang-sam==0.1.0)
  Downloading pytorch_lightning-2.2.1-py3-none-any.whl (801 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 801.6/801.6 kB 50.1 MB/s eta 0:00:00
INFO: pip is looking at multiple versions of transformers to determine which version is compatible with other requirements. This could take a while.
Collecting transformers<5.0.0,>=4.27.4 (from lang-sam==0.1.0)
  Downloading transformers-4.39.0-py3-none-any.whl (8.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.8/8.8 MB 95.6 MB/s eta 0:00:00
  Downloading transformers-4.38.1-py3-none-any.whl (8.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.5/8.5 MB 107.4 MB/s eta 0:00:00
  Downloading transformers-4.38.0-py3-none-any.whl (8.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.5/8.5 MB 98.2 MB/s eta 0:00:00
  Downloading transformers-4.37.2-py3-none-any.whl (8.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.4/8.4 MB 101.7 MB/s eta 0:00:00
  Downloading transformers-4.37.1-py3-none-any.whl (8.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.4/8.4 MB 108.1 MB/s eta 0:00:00
  Downloading transformers-4.37.0-py3-none-any.whl (8.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.4/8.4 MB 99.2 MB/s eta 0:00:00
  Downloading transformers-4.36.2-py3-none-any.whl (8.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.2/8.2 MB 102.3 MB/s eta 0:00:00
INFO: pip is looking at multiple versions of transformers to determine which version is compatible with other requirements. This could take a while.
  Downloading transformers-4.36.1-py3-none-any.whl (8.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.3/8.3 MB 97.4 MB/s eta 0:00:00
  Downloading transformers-4.36.0-py3-none-any.whl (8.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.2/8.2 MB 25.5 MB/s eta 0:00:00
  Downloading transformers-4.35.2-py3-none-any.whl (7.9 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.9/7.9 MB 109.2 MB/s eta 0:00:00
Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers<5.0.0,>=4.27.4->lang-sam==0.1.0) (2023.12.25)
Requirement already satisfied: tokenizers<0.19,>=0.14 in /usr/local/lib/python3.10/dist-packages (from transformers<5.0.0,>=4.27.4->lang-sam==0.1.0) (0.15.2)
Requirement already satisfied: safetensors>=0.3.1 in /usr/local/lib/python3.10/dist-packages (from transformers<5.0.0,>=4.27.4->lang-sam==0.1.0) (0.4.2)
Requirement already satisfied: torchvision in /usr/local/lib/python3.10/dist-packages (from groundingdino@ git+https://github.com/IDEA-Research/GroundingDINO.git->lang-sam==0.1.0) (0.17.1+cu121)
Collecting addict (from groundingdino@ git+https://github.com/IDEA-Research/GroundingDINO.git->lang-sam==0.1.0)
  Downloading addict-2.4.0-py3-none-any.whl (3.8 kB)
Collecting yapf (from groundingdino@ git+https://github.com/IDEA-Research/GroundingDINO.git->lang-sam==0.1.0)
  Downloading yapf-0.40.2-py3-none-any.whl (254 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 254.7/254.7 kB 35.0 MB/s eta 0:00:00
Collecting timm (from groundingdino@ git+https://github.com/IDEA-Research/GroundingDINO.git->lang-sam==0.1.0)
  Downloading timm-0.9.16-py3-none-any.whl (2.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.2/2.2 MB 89.2 MB/s eta 0:00:00
Collecting supervision (from groundingdino@ git+https://github.com/IDEA-Research/GroundingDINO.git->lang-sam==0.1.0)
  Downloading supervision-0.19.0-py3-none-any.whl (97 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 97.1/97.1 kB 16.2 MB/s eta 0:00:00
Requirement already satisfied: pycocotools in /usr/local/lib/python3.10/dist-packages (from groundingdino@ git+https://github.com/IDEA-Research/GroundingDINO.git->lang-sam==0.1.0) (2.0.7)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.10/dist-packages (from altair<6.0,>=4.2.0->gradio<4.0.0,>=3.24.1->lang-sam==0.1.0) (0.4)
Requirement already satisfied: jsonschema>=3.0 in /usr/local/lib/python3.10/dist-packages (from altair<6.0,>=4.2.0->gradio<4.0.0,>=3.24.1->lang-sam==0.1.0) (4.19.2)
Requirement already satisfied: toolz in /usr/local/lib/python3.10/dist-packages (from altair<6.0,>=4.2.0->gradio<4.0.0,>=3.24.1->lang-sam==0.1.0) (0.12.1)
Requirement already satisfied: aiohttp!=4.0.0a0,!=4.0.0a1 in /usr/local/lib/python3.10/dist-packages (from fsspec->gradio-client==0.6.1->gradio<4.0.0,>=3.24.1->lang-sam==0.1.0) (3.9.3)
Requirement already satisfied: setuptools in /usr/local/lib/python3.10/dist-packages (from lightning-utilities<2.0,>=0.8.0->lightning<3.0.0,>=2.0.1->lang-sam==0.1.0) (67.7.2)
Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio<4.0.0,>=3.24.1->lang-sam==0.1.0) (1.2.0)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio<4.0.0,>=3.24.1->lang-sam==0.1.0) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio<4.0.0,>=3.24.1->lang-sam==0.1.0) (4.49.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio<4.0.0,>=3.24.1->lang-sam==0.1.0) (1.4.5)
Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio<4.0.0,>=3.24.1->lang-sam==0.1.0) (3.1.2)
Requirement already satisfied: python-dateutil>=2.7 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio<4.0.0,>=3.24.1->lang-sam==0.1.0) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas<3.0,>=1.0->gradio<4.0.0,>=3.24.1->lang-sam==0.1.0) (2023.4)
Requirement already satisfied: annotated-types>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from pydantic!=1.8,!=1.8.1,!=2.0.0,!=2.0.1,<3.0.0,>=1.7.4->gradio<4.0.0,>=3.24.1->lang-sam==0.1.0) (0.6.0)
Requirement already satisfied: pydantic-core==2.16.3 in /usr/local/lib/python3.10/dist-packages (from pydantic!=1.8,!=1.8.1,!=2.0.0,!=2.0.1,<3.0.0,>=1.7.4->gradio<4.0.0,>=3.24.1->lang-sam==0.1.0) (2.16.3)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests~=2.0->gradio<4.0.0,>=3.24.1->lang-sam==0.1.0) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests~=2.0->gradio<4.0.0,>=3.24.1->lang-sam==0.1.0) (3.6)
Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests~=2.0->gradio<4.0.0,>=3.24.1->lang-sam==0.1.0) (2.0.7)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests~=2.0->gradio<4.0.0,>=3.24.1->lang-sam==0.1.0) (2024.2.2)
Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch<4.0,>=1.13.0->lightning<3.0.0,>=2.0.1->lang-sam==0.1.0) (1.12)
Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch<4.0,>=1.13.0->lightning<3.0.0,>=2.0.1->lang-sam==0.1.0) (3.2.1)
Collecting nvidia-cuda-nvrtc-cu12==12.1.105 (from torch<4.0,>=1.13.0->lightning<3.0.0,>=2.0.1->lang-sam==0.1.0)
  Downloading nvidia_cuda_nvrtc_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (23.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 23.7/23.7 MB 60.3 MB/s eta 0:00:00
Collecting nvidia-cuda-runtime-cu12==12.1.105 (from torch<4.0,>=1.13.0->lightning<3.0.0,>=2.0.1->lang-sam==0.1.0)
  Downloading nvidia_cuda_runtime_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (823 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 823.6/823.6 kB 68.5 MB/s eta 0:00:00
Collecting nvidia-cuda-cupti-cu12==12.1.105 (from torch<4.0,>=1.13.0->lightning<3.0.0,>=2.0.1->lang-sam==0.1.0)
  Downloading nvidia_cuda_cupti_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (14.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 14.1/14.1 MB 90.3 MB/s eta 0:00:00
Collecting nvidia-cudnn-cu12==8.9.2.26 (from torch<4.0,>=1.13.0->lightning<3.0.0,>=2.0.1->lang-sam==0.1.0)
  Downloading nvidia_cudnn_cu12-8.9.2.26-py3-none-manylinux1_x86_64.whl (731.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 731.7/731.7 MB 2.4 MB/s eta 0:00:00
Collecting nvidia-cublas-cu12==12.1.3.1 (from torch<4.0,>=1.13.0->lightning<3.0.0,>=2.0.1->lang-sam==0.1.0)
  Downloading nvidia_cublas_cu12-12.1.3.1-py3-none-manylinux1_x86_64.whl (410.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 410.6/410.6 MB 1.3 MB/s eta 0:00:00
Collecting nvidia-cufft-cu12==11.0.2.54 (from torch<4.0,>=1.13.0->lightning<3.0.0,>=2.0.1->lang-sam==0.1.0)
  Downloading nvidia_cufft_cu12-11.0.2.54-py3-none-manylinux1_x86_64.whl (121.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 121.6/121.6 MB 8.3 MB/s eta 0:00:00
Collecting nvidia-curand-cu12==10.3.2.106 (from torch<4.0,>=1.13.0->lightning<3.0.0,>=2.0.1->lang-sam==0.1.0)
  Downloading nvidia_curand_cu12-10.3.2.106-py3-none-manylinux1_x86_64.whl (56.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 56.5/56.5 MB 9.0 MB/s eta 0:00:00
Collecting nvidia-cusolver-cu12==11.4.5.107 (from torch<4.0,>=1.13.0->lightning<3.0.0,>=2.0.1->lang-sam==0.1.0)
  Downloading nvidia_cusolver_cu12-11.4.5.107-py3-none-manylinux1_x86_64.whl (124.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 124.2/124.2 MB 8.2 MB/s eta 0:00:00
Collecting nvidia-cusparse-cu12==12.1.0.106 (from torch<4.0,>=1.13.0->lightning<3.0.0,>=2.0.1->lang-sam==0.1.0)
  Downloading nvidia_cusparse_cu12-12.1.0.106-py3-none-manylinux1_x86_64.whl (196.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 196.0/196.0 MB 6.4 MB/s eta 0:00:00
Collecting nvidia-nccl-cu12==2.19.3 (from torch<4.0,>=1.13.0->lightning<3.0.0,>=2.0.1->lang-sam==0.1.0)
  Downloading nvidia_nccl_cu12-2.19.3-py3-none-manylinux1_x86_64.whl (166.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 166.0/166.0 MB 7.1 MB/s eta 0:00:00
Collecting nvidia-nvtx-cu12==12.1.105 (from torch<4.0,>=1.13.0->lightning<3.0.0,>=2.0.1->lang-sam==0.1.0)
  Downloading nvidia_nvtx_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (99 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 99.1/99.1 kB 16.5 MB/s eta 0:00:00
Requirement already satisfied: triton==2.2.0 in /usr/local/lib/python3.10/dist-packages (from torch<4.0,>=1.13.0->lightning<3.0.0,>=2.0.1->lang-sam==0.1.0) (2.2.0)
Collecting nvidia-nvjitlink-cu12 (from nvidia-cusolver-cu12==11.4.5.107->torch<4.0,>=1.13.0->lightning<3.0.0,>=2.0.1->lang-sam==0.1.0)
  Downloading nvidia_nvjitlink_cu12-12.4.99-py3-none-manylinux2014_x86_64.whl (21.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 21.1/21.1 MB 61.5 MB/s eta 0:00:00
Requirement already satisfied: click>=7.0 in /usr/local/lib/python3.10/dist-packages (from uvicorn>=0.14.0->gradio<4.0.0,>=3.24.1->lang-sam==0.1.0) (8.1.7)
Collecting h11>=0.8 (from uvicorn>=0.14.0->gradio<4.0.0,>=3.24.1->lang-sam==0.1.0)
  Downloading h11-0.14.0-py3-none-any.whl (58 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.3/58.3 kB 10.1 MB/s eta 0:00:00
Collecting starlette<0.37.0,>=0.36.3 (from fastapi->gradio<4.0.0,>=3.24.1->lang-sam==0.1.0)
  Downloading starlette-0.36.3-py3-none-any.whl (71 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 71.5/71.5 kB 11.2 MB/s eta 0:00:00
Requirement already satisfied: anyio in /usr/local/lib/python3.10/dist-packages (from httpx->gradio<4.0.0,>=3.24.1->lang-sam==0.1.0) (3.7.1)
Collecting httpcore==1.* (from httpx->gradio<4.0.0,>=3.24.1->lang-sam==0.1.0)
  Downloading httpcore-1.0.4-py3-none-any.whl (77 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 77.8/77.8 kB 12.4 MB/s eta 0:00:00
Requirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from httpx->gradio<4.0.0,>=3.24.1->lang-sam==0.1.0) (1.3.1)
Requirement already satisfied: defusedxml<0.8.0,>=0.7.1 in /usr/local/lib/python3.10/dist-packages (from supervision->groundingdino@ git+https://github.com/IDEA-Research/GroundingDINO.git->lang-sam==0.1.0) (0.7.1)
Requirement already satisfied: opencv-python-headless>=4.5.5.64 in /usr/local/lib/python3.10/dist-packages (from supervision->groundingdino@ git+https://github.com/IDEA-Research/GroundingDINO.git->lang-sam==0.1.0) (4.9.0.80)
INFO: pip is looking at multiple versions of supervision to determine which version is compatible with other requirements. This could take a while.
Collecting supervision (from groundingdino@ git+https://github.com/IDEA-Research/GroundingDINO.git->lang-sam==0.1.0)
  Downloading supervision-0.18.0-py3-none-any.whl (86 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 86.7/86.7 kB 5.0 MB/s eta 0:00:00
Requirement already satisfied: scipy<2.0.0,>=1.10.0 in /usr/local/lib/python3.10/dist-packages (from supervision->groundingdino@ git+https://github.com/IDEA-Research/GroundingDINO.git->lang-sam==0.1.0) (1.11.4)
Requirement already satisfied: importlib-metadata>=6.6.0 in /usr/local/lib/python3.10/dist-packages (from yapf->groundingdino@ git+https://github.com/IDEA-Research/GroundingDINO.git->lang-sam==0.1.0) (7.0.2)
Requirement already satisfied: platformdirs>=3.5.1 in /usr/local/lib/python3.10/dist-packages (from yapf->groundingdino@ git+https://github.com/IDEA-Research/GroundingDINO.git->lang-sam==0.1.0) (4.2.0)
Requirement already satisfied: tomli>=2.0.1 in /usr/local/lib/python3.10/dist-packages (from yapf->groundingdino@ git+https://github.com/IDEA-Research/GroundingDINO.git->lang-sam==0.1.0) (2.0.1)
Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec->gradio-client==0.6.1->gradio<4.0.0,>=3.24.1->lang-sam==0.1.0) (1.3.1)
Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec->gradio-client==0.6.1->gradio<4.0.0,>=3.24.1->lang-sam==0.1.0) (23.2.0)
Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec->gradio-client==0.6.1->gradio<4.0.0,>=3.24.1->lang-sam==0.1.0) (1.4.1)
Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec->gradio-client==0.6.1->gradio<4.0.0,>=3.24.1->lang-sam==0.1.0) (6.0.5)
Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec->gradio-client==0.6.1->gradio<4.0.0,>=3.24.1->lang-sam==0.1.0) (1.9.4)
Requirement already satisfied: async-timeout<5.0,>=4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec->gradio-client==0.6.1->gradio<4.0.0,>=3.24.1->lang-sam==0.1.0) (4.0.3)
Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.10/dist-packages (from importlib-metadata>=6.6.0->yapf->groundingdino@ git+https://github.com/IDEA-Research/GroundingDINO.git->lang-sam==0.1.0) (3.18.1)
Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair<6.0,>=4.2.0->gradio<4.0.0,>=3.24.1->lang-sam==0.1.0) (2023.12.1)
Requirement already satisfied: referencing>=0.28.4 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair<6.0,>=4.2.0->gradio<4.0.0,>=3.24.1->lang-sam==0.1.0) (0.33.0)
Requirement already satisfied: rpds-py>=0.7.1 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair<6.0,>=4.2.0->gradio<4.0.0,>=3.24.1->lang-sam==0.1.0) (0.18.0)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.7->matplotlib~=3.0->gradio<4.0.0,>=3.24.1->lang-sam==0.1.0) (1.16.0)
Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio->httpx->gradio<4.0.0,>=3.24.1->lang-sam==0.1.0) (1.2.0)
Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch<4.0,>=1.13.0->lightning<3.0.0,>=2.0.1->lang-sam==0.1.0) (1.3.0)
Building wheels for collected packages: lang-sam, groundingdino, segment-anything, ffmpy
  Building wheel for lang-sam (pyproject.toml) ... done
  Created wheel for lang-sam: filename=lang_sam-0.1.0-py3-none-any.whl size=9855 sha256=0a65f63cbeff59ebc68491319d103eb32024413a1aec5015db0045a90315e452
  Stored in directory: /tmp/pip-ephem-wheel-cache-btl4mu7o/wheels/c7/1d/cd/6f34f8ca4fbc01b7a1f522cf9c4654cbe4fcd4268069d6e739
  Building wheel for groundingdino (setup.py) ... done
  Created wheel for groundingdino: filename=groundingdino-0.1.0-cp310-cp310-linux_x86_64.whl size=2852495 sha256=1286a243c85c716640f71f21ece0cdabae84202514dd2df14e3206c43a7c0d3c
  Stored in directory: /tmp/pip-ephem-wheel-cache-btl4mu7o/wheels/6b/06/d7/b57f601a4df56af41d262a5b1b496359b13c323bf5ef0434b2
  Building wheel for segment-anything (setup.py) ... done
  Created wheel for segment-anything: filename=segment_anything-1.0-py3-none-any.whl size=36590 sha256=8468e1cfbe6fe77023721f284d69d4add42dc0024eb20607767fb7a6afa8d3e4
  Stored in directory: /tmp/pip-ephem-wheel-cache-btl4mu7o/wheels/10/cf/59/9ccb2f0a1bcc81d4fbd0e501680b5d088d690c6cfbc02dc99d
  Building wheel for ffmpy (setup.py) ... done
  Created wheel for ffmpy: filename=ffmpy-0.3.2-py3-none-any.whl size=5584 sha256=a08b2c953f8fc6b1cecf285cc915f34f8bd031b82f59910cde2a7271dcce36c8
  Stored in directory: /root/.cache/pip/wheels/bd/65/9a/671fc6dcde07d4418df0c592f8df512b26d7a0029c2a23dd81
Successfully built lang-sam groundingdino segment-anything ffmpy
Installing collected packages: segment-anything, pydub, ffmpy, addict, websockets, semantic-version, python-multipart, Pillow, orjson, nvidia-nvtx-cu12, nvidia-nvjitlink-cu12, nvidia-nccl-cu12, nvidia-curand-cu12, nvidia-cufft-cu12, nvidia-cuda-runtime-cu12, nvidia-cuda-nvrtc-cu12, nvidia-cuda-cupti-cu12, nvidia-cublas-cu12, lightning-utilities, h11, aiofiles, yapf, uvicorn, starlette, nvidia-cusparse-cu12, nvidia-cudnn-cu12, huggingface-hub, httpcore, supervision, nvidia-cusolver-cu12, httpx, fastapi, transformers, gradio-client, torchmetrics, gradio, timm, pytorch-lightning, lightning, groundingdino, lang-sam
  Attempting uninstall: Pillow
    Found existing installation: Pillow 9.4.0
    Uninstalling Pillow-9.4.0:
      Successfully uninstalled Pillow-9.4.0
  Attempting uninstall: huggingface-hub
    Found existing installation: huggingface-hub 0.20.3
    Uninstalling huggingface-hub-0.20.3:
      Successfully uninstalled huggingface-hub-0.20.3
  Attempting uninstall: transformers
    Found existing installation: transformers 4.38.2
    Uninstalling transformers-4.38.2:
      Successfully uninstalled transformers-4.38.2
Successfully installed Pillow-9.3.0 addict-2.4.0 aiofiles-23.2.1 fastapi-0.110.0 ffmpy-0.3.2 gradio-3.50.2 gradio-client-0.6.1 groundingdino-0.1.0 h11-0.14.0 httpcore-1.0.4 httpx-0.27.0 huggingface-hub-0.16.4 lang-sam-0.1.0 lightning-2.2.1 lightning-utilities-0.11.0 nvidia-cublas-cu12-12.1.3.1 nvidia-cuda-cupti-cu12-12.1.105 nvidia-cuda-nvrtc-cu12-12.1.105 nvidia-cuda-runtime-cu12-12.1.105 nvidia-cudnn-cu12-8.9.2.26 nvidia-cufft-cu12-11.0.2.54 nvidia-curand-cu12-10.3.2.106 nvidia-cusolver-cu12-11.4.5.107 nvidia-cusparse-cu12-12.1.0.106 nvidia-nccl-cu12-2.19.3 nvidia-nvjitlink-cu12-12.4.99 nvidia-nvtx-cu12-12.1.105 orjson-3.9.15 pydub-0.25.1 python-multipart-0.0.9 pytorch-lightning-2.2.1 segment-anything-1.0 semantic-version-2.10.0 starlette-0.36.3 supervision-0.18.0 timm-0.9.16 torchmetrics-1.3.2 transformers-4.35.2 uvicorn-0.29.0 websockets-11.0.3 yapf-0.40.2
In [ ]:
from PIL import Image
from lang_sam import LangSAM

/usr/local/lib/python3.10/dist-packages/transformers/utils/generic.py:441: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
  _torch_pytree._register_pytree_node(
/usr/local/lib/python3.10/dist-packages/transformers/utils/generic.py:309: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
  _torch_pytree._register_pytree_node(

Two examples¶

In [ ]:
img_path = train_path + 'dandelion/' + os.listdir(train_path + 'dandelion/')[1]
In [ ]:
model = LangSAM()
image_pil = Image.open(img_path).convert("RGB")
# Define the text prompt
text_prompt = "flower"
masks, boxes, phrases, logits = model.predict(image_pil, text_prompt)

final text_encoder_type: bert-base-uncased
Model loaded from /root/.cache/huggingface/hub/models--ShilongLiu--GroundingDINO/snapshots/a94c9b567a2a374598f05c584e96798a170c56fb/groundingdino_swinb_cogcoor.pth 
 => _IncompatibleKeys(missing_keys=[], unexpected_keys=['label_enc.weight', 'bert.embeddings.position_ids'])
In [ ]:
np.asarray(image_pil)

ndarray (228, 320, 3) 
array([[[  4,  24,   0],
        [  5,  23,   0],
        [  8,  23,   0],
        ...,
        [ 41,  71,   0],
        [ 44,  74,   0],
        [ 47,  78,   2]],

       [[  4,  24,   0],
        [  5,  23,   0],
        [  8,  23,   0],
        ...,
        [ 44,  74,   0],
        [ 47,  78,   2],
        [ 49,  80,   2]],

       [[  4,  24,   0],
        [  5,  23,   0],
        [  7,  22,   0],
        ...,
        [ 46,  77,   1],
        [ 48,  79,   1],
        [ 50,  81,   1]],

       ...,

       [[ 82, 108,   1],
        [ 69,  96,   0],
        [ 56,  86,   0],
        ...,
        [ 70,  93,   2],
        [ 74,  96,   0],
        [ 79, 101,   1]],

       [[ 83, 109,   2],
        [ 69,  96,   0],
        [ 54,  84,   0],
        ...,
        [ 65,  88,   0],
        [ 71,  92,   0],
        [ 80, 100,   3]],

       [[ 84, 110,   3],
        [ 68,  95,   0],
        [ 53,  83,   0],
        ...,
        [ 61,  83,   0],
        [ 69,  90,   0],
        [ 79,  98,   6]]], dtype=uint8)
In [ ]:
np.asarray(image_pil)[int(boxes[0][1]):int(boxes[0][1])+int(boxes[0][3]), int(boxes[0][0]):int(boxes[0][0])+int(boxes[0][2])]

ndarray (201, 190, 3) 
array([[[  1,  20,   1],
        [  1,  18,   0],
        [  0,  17,   0],
        ...,
        [ 56,  86,   0],
        [ 59,  88,   0],
        [ 62,  91,   0]],

       [[  0,  19,   0],
        [  0,  19,   0],
        [  0,  19,   0],
        ...,
        [ 58,  88,   0],
        [ 61,  90,   0],
        [ 64,  93,   0]],

       [[  0,  19,   0],
        [  0,  19,   0],
        [  0,  19,   0],
        ...,
        [ 62,  93,   0],
        [ 66,  96,   0],
        [ 69,  99,   1]],

       ...,

       [[ 67,  97,   1],
        [ 60,  89,   0],
        [ 54,  82,   0],
        ...,
        [ 97, 114,   0],
        [ 98, 116,   0],
        [100, 118,   0]],

       [[ 67,  94,   0],
        [ 61,  87,   0],
        [ 53,  81,   0],
        ...,
        [ 93, 110,   0],
        [ 94, 111,   0],
        [ 97, 115,   0]],

       [[ 69,  96,   0],
        [ 57,  87,   1],
        [ 53,  81,   4],
        ...,
        [ 90, 105,   0],
        [ 91, 109,   1],
        [ 92, 110,   2]]], dtype=uint8)
In [ ]:
img_path = train_path + 'dandelion/' + os.listdir(train_path + 'dandelion/')[2]
image_pil = Image.open(img_path).convert("RGB")
In [ ]:
np.asarray(image_pil)

ndarray (241, 320, 3) 
array([[[36, 45, 24],
        [35, 44, 23],
        [34, 44, 20],
        ...,
        [44, 54, 30],
        [44, 54, 30],
        [43, 53, 29]],

       [[35, 44, 23],
        [35, 44, 23],
        [34, 44, 20],
        ...,
        [44, 54, 30],
        [44, 54, 30],
        [43, 53, 29]],

       [[35, 43, 20],
        [35, 43, 20],
        [35, 43, 20],
        ...,
        [44, 54, 30],
        [43, 53, 29],
        [43, 53, 29]],

       ...,

       [[43, 60,  6],
        [48, 65, 10],
        [53, 71, 13],
        ...,
        [21, 29,  6],
        [21, 29,  6],
        [22, 30,  7]],

       [[46, 60,  7],
        [50, 64, 11],
        [53, 70, 15],
        ...,
        [20, 28,  7],
        [21, 29,  6],
        [21, 29,  6]],

       [[45, 59,  8],
        [49, 64,  9],
        [54, 70,  8],
        ...,
        [21, 29,  8],
        [20, 28,  7],
        [20, 28,  7]]], dtype=uint8)
In [ ]:
masks, boxes, phrases, logits = model.predict(image_pil, text_prompt)

/usr/local/lib/python3.10/dist-packages/transformers/modeling_utils.py:907: FutureWarning: The `device` argument is deprecated and will be removed in v5 of Transformers.
  warnings.warn(
/usr/local/lib/python3.10/dist-packages/torch/utils/checkpoint.py:460: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
  warnings.warn(
/usr/local/lib/python3.10/dist-packages/torch/utils/checkpoint.py:90: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(
In [ ]:
np.asarray(image_pil)[int(boxes[0][1]):int(boxes[0][1])+int(boxes[0][3]), int(boxes[0][0]):int(boxes[0][0])+int(boxes[0][2])]

ndarray (206, 221, 3) 
array([[[ 31,  41,  14],
        [ 31,  41,  14],
        [ 31,  41,  14],
        ...,
        [ 17,  26,   7],
        [ 17,  26,   7],
        [ 17,  26,   7]],

       [[ 31,  41,  14],
        [ 31,  41,  14],
        [ 31,  41,  14],
        ...,
        [ 16,  25,   6],
        [ 17,  26,   7],
        [ 17,  26,   7]],

       [[ 31,  41,  14],
        [ 31,  41,  14],
        [ 30,  40,  13],
        ...,
        [ 16,  25,   6],
        [ 16,  25,   6],
        [ 17,  26,   7]],

       ...,

       [[153, 145,  73],
        [142, 137,  69],
        [116, 115,  58],
        ...,
        [ 21,  29,   6],
        [ 21,  29,   6],
        [ 22,  30,   7]],

       [[150, 142,  70],
        [136, 131,  63],
        [109, 110,  50],
        ...,
        [ 20,  28,   7],
        [ 21,  29,   6],
        [ 21,  29,   6]],

       [[147, 141,  65],
        [131, 128,  57],
        [ 98, 102,  41],
        ...,
        [ 21,  29,   8],
        [ 20,  28,   7],
        [ 20,  28,   7]]], dtype=uint8)

Segment Train and Test¶

Now, we apply the segmentation to all of our data

In [15]:
from tqdm import tqdm
import warnings
import os
import numpy as np
from PIL import Image
import matplotlib.pyplot as plt
import re
import random
warnings.filterwarnings("ignore")
classes = os.listdir(train_path)
In [ ]:
classes

['dandelion', 'tulips', 'daisy', 'sunflowers', 'roses']

Train¶

In [ ]:
base_path = 'C:/Users/david/Desktop/DeepLearning/flower_photos/Train/'
save_path = 'C:/Users/david/Desktop/DeepLearning/flower_cropped/Train/'
In [ ]:
for flower_class in classes:
  img_list = os.listdir(base_path + flower_class)
  already_saved = []
  for el in os.listdir(save_path + flower_class):
    # remove '.jpg'
    el = ''.join(el.split('.')[:-1])
    if el[-2:] == '_0':
      already_saved.append(el[:-2] + '.jpg')
  for img_path in tqdm(img_list):
    if img_path in already_saved:
      continue
    image_pil = Image.open(base_path + flower_class + '/' + img_path).convert("RGB")
    masks, boxes, phrases, logits = model.predict(image_pil, text_prompt)
    for i in range(len(boxes)):
      cropped_img = np.asarray(image_pil)[int(boxes[i][1]):int(boxes[i][1])+int(boxes[i][3]), int(boxes[i][0]):int(boxes[i][0])+int(boxes[i][2])]
      final_img = Image.fromarray(cropped_img)
      final_img.save(save_path + flower_class + '/' + ''.join(img_path.split('.')[:-1]) + '_' + str(i) + '.jpg')

100%|██████████| 701/701 [00:00<00:00, 1500.13it/s]
100%|██████████| 632/632 [05:19<00:00,  1.98it/s]
100%|██████████| 508/508 [00:00<00:00, 99710.16it/s]
100%|██████████| 561/561 [00:00<00:00, 77424.39it/s]
100%|██████████| 534/534 [00:18<00:00, 28.28it/s]

Test¶

In [ ]:
base_path = 'C:/Users/david/Desktop/DeepLearning/flower_photos/Test/'
save_path = 'C:/Users/david/Desktop/DeepLearning/flower_cropped/Test/'
In [ ]:
for flower_class in classes:
  img_list = os.listdir(base_path + flower_class)
  already_saved = []
  for el in os.listdir(save_path + flower_class):
    # remove '.jpg'
    el = ''.join(el.split('.')[:-1])
    if el[-2:] == '_0':
      already_saved.append(el[:-2] + '.jpg')
  for img_path in tqdm(img_list):
    if img_path in already_saved:
      continue
    image_pil = Image.open(base_path + flower_class + '/' + img_path).convert("RGB")
    masks, boxes, phrases, logits = model.predict(image_pil, text_prompt)
    for i in range(len(boxes)):
      # There could be more than a flower in a image!
      cropped_img = np.asarray(image_pil)[int(boxes[i][1]):int(boxes[i][1])+int(boxes[i][3]), int(boxes[i][0]):int(boxes[i][0])+int(boxes[i][2])]
      final_img = Image.fromarray(cropped_img)
      final_img.save(save_path + flower_class + '/' + ''.join(img_path.split('.')[:-1]) + '_' + str(i) + '.jpg')

100%|██████████| 197/197 [00:02<00:00, 93.82it/s]
100%|██████████| 167/167 [00:00<00:00, 119469.34it/s]
100%|██████████| 125/125 [00:00<00:00, 374224.13it/s]
100%|██████████| 138/138 [00:00<00:00, 289406.98it/s]
100%|██████████| 107/107 [00:01<00:00, 58.98it/s]

Dimensionality check¶

Let's check that there isn't anything strange with the pictures.

While segmenting the images, we noticed that some pciture have strange dimensions. Here is an example:

In [49]:
image_pil = Image.open(train_path + 'daisy/102841525_bd6628ae3c_16.jpg').convert("RGB")

# Display the image
plt.imshow(np.asarray(image_pil))
plt.axis('off')  # Turn off axis
plt.show()
In [50]:
clas_diz['daisy']['102841525_bd6628ae3c_16.jpg']
Out[50]:
(11, 165, 3)

To avoid that this type of images get into the training dataset, we are going to remove every image that has one dimension that is shorter than 15pixels (this is an arbitrary measure).

In [ ]:
# Get the dimensions of all images in train set
clas_diz = {}
for clas in classes:
    diz = {}
    for img_path in tqdm(os.listdir(train_path + clas)):
        image_pil = Image.open(train_path + clas + '/' + img_path).convert("RGB")
        diz[img_path] = np.asarray(image_pil).shape
    clas_diz[clas] = diz

100%|██████████| 2546/2546 [00:11<00:00, 220.33it/s]
100%|██████████| 1301/1301 [00:05<00:00, 225.42it/s]
100%|██████████| 1518/1518 [00:06<00:00, 219.03it/s]
100%|██████████| 2282/2282 [00:09<00:00, 232.48it/s]
100%|██████████| 3054/3054 [00:12<00:00, 235.73it/s]
In [22]:
test_path = base_path + 'Test/'
# Get the dimensions of all images in test set
clas_diz_test = {}
for clas in classes:
    diz = {}
    for img_path in tqdm(os.listdir(test_path + clas)):
        image_pil = Image.open(test_path + clas + '/' + img_path).convert("RGB")
        diz[img_path] = np.asarray(image_pil).shape
    clas_diz_test[clas] = diz

100%|██████████| 699/699 [00:00<00:00, 2583.21it/s]
100%|██████████| 399/399 [00:00<00:00, 2328.88it/s]
100%|██████████| 263/263 [00:00<00:00, 2351.93it/s]
100%|██████████| 586/586 [00:00<00:00, 2340.42it/s]
100%|██████████| 688/688 [00:00<00:00, 2509.84it/s]
In [59]:
delete_diz = clas_diz.copy()
In [67]:
# Insert all of the training images with a dimension lower that 15px into the "delete_diz" dictionary
delete_diz = {}
for clas in classes:
    delete_diz[clas] = []
    for key, value in clas_diz[clas].items():
        if value[0] <= 15 or value[1] <= 15:
            delete_diz[clas].append(key)
In [68]:
# How many images are we going to delete from training set?
diz = {}
for clas in classes:
    diz[clas] = len(delete_diz[clas])

diz
Out[68]:
{'daisy': 69, 'dandelion': 26, 'roses': 14, 'sunflowers': 53, 'tulips': 68}
In [69]:
# Delete all the images in the "delete_diz" dictionary
for clas in classes:
    for img_path in delete_diz[clas]:
        os.remove(train_path + clas + '/' + img_path)
In [23]:
delete_diz = clas_diz_test.copy()
In [24]:
# Insert all of the test images with a dimension lower that 15px into the "delete_diz" dictionary
delete_diz = {}
for clas in classes:
    delete_diz[clas] = []
    for key, value in clas_diz_test[clas].items():
        if value[0] <= 15 or value[1] <= 15:
            delete_diz[clas].append(key)
In [25]:
# How many images are we going to delete from test set?
diz = {}
for clas in classes:
    diz[clas] = len(delete_diz[clas])

diz
Out[25]:
{'daisy': 17, 'dandelion': 7, 'roses': 2, 'sunflowers': 18, 'tulips': 17}
In [27]:
# Delete all the images in the "delete_diz" dictionary
for clas in classes:
    for img_path in delete_diz[clas]:
        os.remove(test_path + clas + '/' + img_path)

How many images have we generated?¶

Since in each picture there can be more than one flower, the dimensinoality of the dataset have changed, let's check how much.

In [31]:
train_path = 'C:/Users/david/Desktop/DeepLearning/flower_cropped/Train/'
subD = os.listdir(train_path)
dizTrain = {}
# For each class subdirectory, insert in the dictionary dizTrain the number of element in that subdirectory
for dir in subD:
    dizTrain[dir] = len(os.listdir(train_path + dir))
In [32]:
# Number of observations of each class in train set
dizTrain
Out[32]:
{'daisy': 2477,
 'dandelion': 1275,
 'roses': 1504,
 'sunflowers': 2229,
 'tulips': 2986}
In [33]:
subD = os.listdir(test_path)
dizTest = {}
# For each class subdirectory, insert in the dictionary dizTrain the number of element in that subdirectory
for dir in subD:
    dizTest[dir] = len(os.listdir(test_path + dir))
In [34]:
# Number of observations of each class in train set
dizTest
Out[34]:
{'daisy': 682,
 'dandelion': 392,
 'roses': 261,
 'sunflowers': 568,
 'tulips': 671}
In [35]:
plt.bar(dizTrain.keys(), dizTrain.values(), color='b', alpha=0.5, label='Train')
plt.bar(dizTest.keys(), dizTest.values(), color='r', alpha=0.5, label='Test')

# Adding labels and title
plt.xlabel('N. of Observations')
plt.ylabel('Classes')
plt.title('N. of Observations per class')
plt.legend()

# Show plot
plt.show()

The segmentation increased the difference between the classes. We are firstly going to run the models with this data, then we are going to try balancing them.

Build the test dataset¶

For what regards the test set, we now have multiple cropped picture coming from the same original image. This could cause some problem in the evaluation of the model, since there would be some original image that appears more times then other in the test set (those with more flowers detected from the segmentation model).
To solve this issue we are going to build 3 different test sets, one with its own pro and cons:

  1. The original test set with uncropped images. It is going to be used to evaluate the model performances to generalized even to the raw, unsegmented, original images.
  2. The unbalances test set. It is going to be used to evaluate the model performances on all the flowers that the segmented model has detected.
  3. A balanced segmented test set. Composed of one segmented image for each original picture. This resolves the problem of the unbalanced test set (there will be only one segmented image for each original picture), but it does so by selecting one random segmented image (the first one for simplicity) from each original picture, and this adds randomness.
In [38]:
test_path_balanced = base_path + 'TestBalanced/'
In [39]:
classes
Out[39]:
['daisy', 'dandelion', 'roses', 'sunflowers', 'tulips']
In [43]:
# We are simply going to mantain only the first segmented image for each original picture
# That is, we are going to mantain all those images that end with "_0.jpg"
for clas in classes:
    for img in os.listdir(test_path_balanced + clas):
        if img[-6:] != '_0.jpg':
            os.remove(test_path_balanced + clas + '/' + img)
In [44]:
# Check how many images for each class
subD = os.listdir(test_path_balanced)
dizTestB = {}
# For each class subdirectory, insert in the dictionary dizTrain the number of element in that subdirectory
for dir in subD:
    dizTestB[dir] = len(os.listdir(test_path_balanced + dir))
dizTestB
Out[44]:
{'daisy': 125,
 'dandelion': 196,
 'roses': 106,
 'sunflowers': 138,
 'tulips': 167}

Load test data¶

In [4]:
# First test set (original)
base_path = 'C:/Users/david/Desktop/DeepLearning/flower_photos/'
test_ds = keras.preprocessing.image_dataset_from_directory(base_path +'Test',
                                                            image_size=(224,224),
                                                            batch_size=128,
                                                            label_mode='categorical',
                                                            shuffle=False,
                                                            seed=1)
# Second test set (segmented)
base_path = 'C:/Users/david/Desktop/DeepLearning/flower_cropped/'
test_ds_cropped = keras.preprocessing.image_dataset_from_directory(base_path +'Test',
                                                            image_size=(224,224),
                                                            batch_size=128,
                                                            label_mode='categorical',
                                                            shuffle=False,
                                                            seed=1)

# Third test set (segmented balanced)
test_ds_cropped_balanced = keras.preprocessing.image_dataset_from_directory(base_path +'TestBalanced',
                                                            image_size=(224,224),
                                                            batch_size=128,
                                                            label_mode='categorical',
                                                            shuffle=False,
                                                            seed=1)

Found 734 files belonging to 5 classes.
Found 2574 files belonging to 5 classes.
Found 732 files belonging to 5 classes.

Note: the 2 photo difference between original and segmented balanced is probably due to the fact that the segmentation model did not find any flower to segment in those two images, thus not producing any segmented image.

CNN with Segmented Data¶

In [3]:
base_path = 'C:/Users/david/Desktop/DeepLearning/flower_cropped/'

train_ds = keras.preprocessing.image_dataset_from_directory(base_path + 'Train',
                                                            image_size=(224,224),
                                                            batch_size=32,
                                                            label_mode='categorical',
                                                            subset='training',
                                                            validation_split = 0.3,
                                                            shuffle=True,
                                                            seed=1)

val_ds = keras.preprocessing.image_dataset_from_directory(base_path + 'Train',
                                                            image_size=(224,224),
                                                            batch_size=32,
                                                            label_mode='categorical',
                                                            subset='validation',
                                                            validation_split = 0.3,
                                                            shuffle=True,
                                                            seed=1)

test_ds_cropped = keras.preprocessing.image_dataset_from_directory(base_path +'Test',
                                                            image_size=(224,224),
                                                            batch_size=128,
                                                            label_mode='categorical',
                                                            shuffle=False,
                                                            seed=1)

# Data loader with (slightly) larger images
train_ds_256 = keras.preprocessing.image_dataset_from_directory( base_path + 'Train',
                                                            image_size=(256,256),
                                                            batch_size=32,
                                                            label_mode='categorical',
                                                            subset='training',
                                                            validation_split = 0.3,
                                                            shuffle=True,
                                                            seed=1)

num_classes = 5

Found 10471 files belonging to 5 classes.
Using 7330 files for training.
Found 10471 files belonging to 5 classes.
Using 3141 files for validation.
Found 800 files belonging to 5 classes.
Found 10471 files belonging to 5 classes.
Using 7330 files for training.
In [4]:
train_ds_256.file_paths == train_ds.file_paths
Out[4]:
True

Unbalanced Data¶

First model¶

We are firstly going to evaluate the performances of the first model of the two that achieved the best performances with the non-segmented images:

In [15]:
# Architecture definition of first model.
inputs = keras.Input((None,None,3))

x = inputs

x = keras.layers.RandomCrop(224, 224)(x)
x = keras.layers.RandomFlip(mode='horizontal')(x)
x = keras.layers.RandomContrast(0.25)(x)

x = keras.layers.Conv2D(32, 3, padding="same", kernel_regularizer=tf.keras.regularizers.l2(0.001))(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.MaxPooling2D(3, strides=3, padding="same")(x)

x = keras.layers.Conv2D(64, 3, padding="same", kernel_regularizer=tf.keras.regularizers.l2(0.001))(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.MaxPooling2D(3, strides=3, padding="same")(x)

x = keras.layers.Conv2D(128, 3, padding="same", kernel_regularizer=tf.keras.regularizers.l2(0.001))(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)

x = keras.layers.GlobalMaxPooling2D()(x)

outputs = keras.layers.Dense(num_classes, activation="softmax", kernel_regularizer=tf.keras.regularizers.l2(0.001))(x)
net = keras.Model(inputs, outputs)
In [16]:
callback = keras.callbacks.EarlyStopping(monitor='val_loss', patience=10)
In [17]:
# Compile the model for training
net.compile(loss=keras.losses.categorical_crossentropy,
            optimizer=keras.optimizers.RMSprop(learning_rate=0.0001),
            metrics=['accuracy'])
In [19]:
# Model training (v4)
history = net.fit(train_ds_256,
          epochs=80,
          validation_data=val_ds,
          callbacks = [callback])

Epoch 1/80
230/230 [==============================] - 25s 106ms/step - loss: 1.1631 - accuracy: 0.5985 - val_loss: 1.0932 - val_accuracy: 0.6339
Epoch 2/80
230/230 [==============================] - 24s 106ms/step - loss: 0.9748 - accuracy: 0.6779 - val_loss: 0.9130 - val_accuracy: 0.7007
Epoch 3/80
230/230 [==============================] - 24s 105ms/step - loss: 0.9172 - accuracy: 0.7064 - val_loss: 0.8685 - val_accuracy: 0.7176
Epoch 4/80
230/230 [==============================] - 24s 102ms/step - loss: 0.8637 - accuracy: 0.7252 - val_loss: 0.8345 - val_accuracy: 0.7278
Epoch 5/80
230/230 [==============================] - 23s 102ms/step - loss: 0.8293 - accuracy: 0.7412 - val_loss: 0.8290 - val_accuracy: 0.7297
Epoch 6/80
230/230 [==============================] - 24s 102ms/step - loss: 0.8073 - accuracy: 0.7486 - val_loss: 0.8244 - val_accuracy: 0.7396
Epoch 7/80
230/230 [==============================] - 24s 102ms/step - loss: 0.7812 - accuracy: 0.7607 - val_loss: 0.7926 - val_accuracy: 0.7482
Epoch 8/80
230/230 [==============================] - 24s 104ms/step - loss: 0.7557 - accuracy: 0.7640 - val_loss: 0.7653 - val_accuracy: 0.7705
Epoch 9/80
230/230 [==============================] - 28s 123ms/step - loss: 0.7419 - accuracy: 0.7720 - val_loss: 0.7541 - val_accuracy: 0.7593
Epoch 10/80
230/230 [==============================] - 31s 133ms/step - loss: 0.7297 - accuracy: 0.7782 - val_loss: 0.7339 - val_accuracy: 0.7752
Epoch 11/80
230/230 [==============================] - 34s 149ms/step - loss: 0.7135 - accuracy: 0.7847 - val_loss: 0.7539 - val_accuracy: 0.7539
Epoch 12/80
230/230 [==============================] - 38s 163ms/step - loss: 0.7104 - accuracy: 0.7844 - val_loss: 0.7312 - val_accuracy: 0.7752
Epoch 13/80
230/230 [==============================] - 41s 177ms/step - loss: 0.6818 - accuracy: 0.7967 - val_loss: 0.7220 - val_accuracy: 0.7784
Epoch 14/80
230/230 [==============================] - 41s 177ms/step - loss: 0.6823 - accuracy: 0.7973 - val_loss: 0.7351 - val_accuracy: 0.7685
Epoch 15/80
230/230 [==============================] - 43s 185ms/step - loss: 0.6724 - accuracy: 0.8080 - val_loss: 0.6969 - val_accuracy: 0.7985
Epoch 16/80
230/230 [==============================] - 44s 192ms/step - loss: 0.6659 - accuracy: 0.8029 - val_loss: 0.6942 - val_accuracy: 0.7905
Epoch 17/80
230/230 [==============================] - 46s 199ms/step - loss: 0.6416 - accuracy: 0.8135 - val_loss: 0.6795 - val_accuracy: 0.8013
Epoch 18/80
230/230 [==============================] - 48s 207ms/step - loss: 0.6406 - accuracy: 0.8104 - val_loss: 0.6849 - val_accuracy: 0.7918
Epoch 19/80
230/230 [==============================] - 48s 210ms/step - loss: 0.6350 - accuracy: 0.8192 - val_loss: 0.6805 - val_accuracy: 0.7931
Epoch 20/80
230/230 [==============================] - 49s 214ms/step - loss: 0.6273 - accuracy: 0.8183 - val_loss: 0.7068 - val_accuracy: 0.7937
Epoch 21/80
230/230 [==============================] - 50s 216ms/step - loss: 0.6212 - accuracy: 0.8232 - val_loss: 0.7084 - val_accuracy: 0.7784
Epoch 22/80
230/230 [==============================] - 51s 223ms/step - loss: 0.6167 - accuracy: 0.8186 - val_loss: 0.6969 - val_accuracy: 0.7822
Epoch 23/80
230/230 [==============================] - 53s 231ms/step - loss: 0.6101 - accuracy: 0.8247 - val_loss: 0.6690 - val_accuracy: 0.8042
Epoch 24/80
230/230 [==============================] - 54s 235ms/step - loss: 0.5970 - accuracy: 0.8303 - val_loss: 0.7248 - val_accuracy: 0.7720
Epoch 25/80
230/230 [==============================] - 55s 240ms/step - loss: 0.5948 - accuracy: 0.8274 - val_loss: 0.6384 - val_accuracy: 0.8192
Epoch 26/80
230/230 [==============================] - 56s 242ms/step - loss: 0.5900 - accuracy: 0.8292 - val_loss: 0.6724 - val_accuracy: 0.7943
Epoch 27/80
230/230 [==============================] - 56s 241ms/step - loss: 0.5828 - accuracy: 0.8334 - val_loss: 0.6306 - val_accuracy: 0.8201
Epoch 28/80
230/230 [==============================] - 54s 234ms/step - loss: 0.5790 - accuracy: 0.8344 - val_loss: 0.6290 - val_accuracy: 0.8147
Epoch 29/80
230/230 [==============================] - 54s 233ms/step - loss: 0.5747 - accuracy: 0.8408 - val_loss: 0.6589 - val_accuracy: 0.8010
Epoch 30/80
230/230 [==============================] - 54s 233ms/step - loss: 0.5661 - accuracy: 0.8431 - val_loss: 0.6278 - val_accuracy: 0.8118
Epoch 31/80
230/230 [==============================] - 54s 232ms/step - loss: 0.5591 - accuracy: 0.8437 - val_loss: 0.6197 - val_accuracy: 0.8208
Epoch 32/80
230/230 [==============================] - 53s 231ms/step - loss: 0.5521 - accuracy: 0.8512 - val_loss: 0.6221 - val_accuracy: 0.8182
Epoch 33/80
230/230 [==============================] - 52s 228ms/step - loss: 0.5552 - accuracy: 0.8495 - val_loss: 0.6406 - val_accuracy: 0.8036
Epoch 34/80
230/230 [==============================] - 52s 228ms/step - loss: 0.5483 - accuracy: 0.8491 - val_loss: 0.6090 - val_accuracy: 0.8208
Epoch 35/80
230/230 [==============================] - 52s 227ms/step - loss: 0.5419 - accuracy: 0.8447 - val_loss: 0.6025 - val_accuracy: 0.8239
Epoch 36/80
230/230 [==============================] - 52s 227ms/step - loss: 0.5392 - accuracy: 0.8506 - val_loss: 0.5911 - val_accuracy: 0.8325
Epoch 37/80
230/230 [==============================] - 52s 227ms/step - loss: 0.5330 - accuracy: 0.8562 - val_loss: 0.6345 - val_accuracy: 0.8179
Epoch 38/80
230/230 [==============================] - 52s 227ms/step - loss: 0.5281 - accuracy: 0.8569 - val_loss: 0.6234 - val_accuracy: 0.8080
Epoch 39/80
230/230 [==============================] - 52s 227ms/step - loss: 0.5174 - accuracy: 0.8587 - val_loss: 0.5965 - val_accuracy: 0.8290
Epoch 40/80
230/230 [==============================] - 52s 227ms/step - loss: 0.5208 - accuracy: 0.8621 - val_loss: 0.5910 - val_accuracy: 0.8262
Epoch 41/80
230/230 [==============================] - 52s 228ms/step - loss: 0.5139 - accuracy: 0.8610 - val_loss: 0.6059 - val_accuracy: 0.8182
Epoch 42/80
230/230 [==============================] - 52s 227ms/step - loss: 0.5116 - accuracy: 0.8658 - val_loss: 0.6542 - val_accuracy: 0.7959
Epoch 43/80
230/230 [==============================] - 52s 228ms/step - loss: 0.5117 - accuracy: 0.8628 - val_loss: 0.6079 - val_accuracy: 0.8236
Epoch 44/80
230/230 [==============================] - 53s 232ms/step - loss: 0.4973 - accuracy: 0.8651 - val_loss: 0.5914 - val_accuracy: 0.8287
Epoch 45/80
230/230 [==============================] - 56s 245ms/step - loss: 0.4982 - accuracy: 0.8685 - val_loss: 0.6124 - val_accuracy: 0.8233
Epoch 46/80
230/230 [==============================] - 56s 243ms/step - loss: 0.4962 - accuracy: 0.8674 - val_loss: 0.5708 - val_accuracy: 0.8376
Epoch 47/80
230/230 [==============================] - 53s 230ms/step - loss: 0.4900 - accuracy: 0.8715 - val_loss: 0.6228 - val_accuracy: 0.8093
Epoch 48/80
230/230 [==============================] - 53s 230ms/step - loss: 0.4934 - accuracy: 0.8716 - val_loss: 0.6341 - val_accuracy: 0.7972
Epoch 49/80
230/230 [==============================] - 53s 229ms/step - loss: 0.4859 - accuracy: 0.8688 - val_loss: 0.5926 - val_accuracy: 0.8246
Epoch 50/80
230/230 [==============================] - 53s 229ms/step - loss: 0.4862 - accuracy: 0.8683 - val_loss: 0.5629 - val_accuracy: 0.8395
Epoch 51/80
230/230 [==============================] - 53s 229ms/step - loss: 0.4804 - accuracy: 0.8693 - val_loss: 0.5846 - val_accuracy: 0.8211
Epoch 52/80
230/230 [==============================] - 53s 231ms/step - loss: 0.4735 - accuracy: 0.8774 - val_loss: 0.5777 - val_accuracy: 0.8325
Epoch 53/80
230/230 [==============================] - 53s 231ms/step - loss: 0.4720 - accuracy: 0.8744 - val_loss: 0.6090 - val_accuracy: 0.8230
Epoch 54/80
230/230 [==============================] - 53s 231ms/step - loss: 0.4688 - accuracy: 0.8827 - val_loss: 0.6113 - val_accuracy: 0.8166
Epoch 55/80
230/230 [==============================] - 53s 229ms/step - loss: 0.4638 - accuracy: 0.8799 - val_loss: 0.5730 - val_accuracy: 0.8399
Epoch 56/80
230/230 [==============================] - 53s 230ms/step - loss: 0.4571 - accuracy: 0.8850 - val_loss: 0.5575 - val_accuracy: 0.8399
Epoch 57/80
230/230 [==============================] - 53s 229ms/step - loss: 0.4554 - accuracy: 0.8823 - val_loss: 0.5588 - val_accuracy: 0.8360
Epoch 58/80
230/230 [==============================] - 53s 229ms/step - loss: 0.4548 - accuracy: 0.8835 - val_loss: 0.5928 - val_accuracy: 0.8233
Epoch 59/80
230/230 [==============================] - 53s 229ms/step - loss: 0.4486 - accuracy: 0.8843 - val_loss: 0.5451 - val_accuracy: 0.8469
Epoch 60/80
230/230 [==============================] - 53s 230ms/step - loss: 0.4565 - accuracy: 0.8842 - val_loss: 0.5837 - val_accuracy: 0.8399
Epoch 61/80
230/230 [==============================] - 53s 230ms/step - loss: 0.4557 - accuracy: 0.8760 - val_loss: 0.5691 - val_accuracy: 0.8313
Epoch 62/80
230/230 [==============================] - 53s 231ms/step - loss: 0.4421 - accuracy: 0.8903 - val_loss: 0.5564 - val_accuracy: 0.8408
Epoch 63/80
230/230 [==============================] - 53s 232ms/step - loss: 0.4423 - accuracy: 0.8857 - val_loss: 0.5421 - val_accuracy: 0.8456
Epoch 64/80
230/230 [==============================] - 53s 232ms/step - loss: 0.4374 - accuracy: 0.8864 - val_loss: 0.5935 - val_accuracy: 0.8198
Epoch 65/80
230/230 [==============================] - 53s 230ms/step - loss: 0.4400 - accuracy: 0.8887 - val_loss: 0.5959 - val_accuracy: 0.8246
Epoch 66/80
230/230 [==============================] - 53s 230ms/step - loss: 0.4430 - accuracy: 0.8877 - val_loss: 0.5677 - val_accuracy: 0.8303
Epoch 67/80
230/230 [==============================] - 53s 229ms/step - loss: 0.4288 - accuracy: 0.8936 - val_loss: 0.5464 - val_accuracy: 0.8367
Epoch 68/80
230/230 [==============================] - 54s 235ms/step - loss: 0.4208 - accuracy: 0.8985 - val_loss: 0.5321 - val_accuracy: 0.8415
Epoch 69/80
230/230 [==============================] - 54s 232ms/step - loss: 0.4255 - accuracy: 0.8922 - val_loss: 0.5450 - val_accuracy: 0.8459
Epoch 70/80
230/230 [==============================] - 53s 229ms/step - loss: 0.4206 - accuracy: 0.8930 - val_loss: 0.5328 - val_accuracy: 0.8465
Epoch 71/80
230/230 [==============================] - 53s 229ms/step - loss: 0.4281 - accuracy: 0.8880 - val_loss: 0.5358 - val_accuracy: 0.8408
Epoch 72/80
230/230 [==============================] - 53s 230ms/step - loss: 0.4193 - accuracy: 0.8958 - val_loss: 0.6141 - val_accuracy: 0.8144
Epoch 73/80
230/230 [==============================] - 53s 229ms/step - loss: 0.4175 - accuracy: 0.8940 - val_loss: 0.5481 - val_accuracy: 0.8405
Epoch 74/80
230/230 [==============================] - 53s 229ms/step - loss: 0.4099 - accuracy: 0.8984 - val_loss: 0.5456 - val_accuracy: 0.8376
Epoch 75/80
230/230 [==============================] - 53s 229ms/step - loss: 0.4117 - accuracy: 0.8981 - val_loss: 0.5445 - val_accuracy: 0.8408
Epoch 76/80
230/230 [==============================] - 53s 230ms/step - loss: 0.4063 - accuracy: 0.9004 - val_loss: 0.5835 - val_accuracy: 0.8319
Epoch 77/80
230/230 [==============================] - 53s 230ms/step - loss: 0.4071 - accuracy: 0.8990 - val_loss: 0.5553 - val_accuracy: 0.8408
Epoch 78/80
230/230 [==============================] - 53s 229ms/step - loss: 0.3980 - accuracy: 0.9048 - val_loss: 0.5361 - val_accuracy: 0.8459
In [20]:
# Visualization of the learning curves

plt.subplot(1, 2, 1)
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.ylim([0.0, 2.0])
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.legend(['train', 'valid'])

plt.subplot(1, 2, 2)
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.ylim([0.5, 1.0])
plt.xlabel('Epoch')
plt.ylabel('Accuracy')
plt.legend(['train', 'valid'])
Out[20]:
<matplotlib.legend.Legend at 0x1c689c5aac0>
In [21]:
net.save('models/Segmented1RMS.keras')
In [5]:
# Evaluate on the 3 test dataset:
net = keras.models.load_model("models/Segmented1RMS.keras")
net.evaluate(test_ds)
net.evaluate(test_ds_cropped)
net.evaluate(test_ds_cropped_balanced)

6/6 [==============================] - 5s 475ms/step - loss: 0.7436 - accuracy: 0.7956
21/21 [==============================] - 4s 179ms/step - loss: 0.6731 - accuracy: 0.8085
6/6 [==============================] - 2s 446ms/step - loss: 0.6117 - accuracy: 0.8333
Out[5]:
[0.6117007732391357, 0.8333333134651184]

None of the three accuracy resulted to be higher than the test accuracy we achieved with the model with unsegmented data (83.65%).

Second model¶

The second model between the two best built with the unsegmented data:

In [22]:
# Architecture definition
inputs = keras.Input((None,None,3))

x = inputs

x = keras.layers.RandomCrop(224, 224)(x)
x = keras.layers.RandomFlip(mode='horizontal')(x)
x = keras.layers.RandomContrast(0.25)(x)

x = keras.layers.Conv2D(64, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.MaxPooling2D(3, strides=3, padding="same")(x)
x = keras.layers.Dropout(0.1)(x)

x = keras.layers.Conv2D(128, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.MaxPooling2D(3, strides=3, padding="same")(x)
x = keras.layers.Dropout(0.1)(x)

x = keras.layers.Conv2D(256, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.Dropout(0.1)(x)

x = keras.layers.GlobalMaxPooling2D()(x)

outputs = keras.layers.Dense(1024, activation="softmax")(x)
x = keras.layers.Dropout(0.2)(x)

outputs = keras.layers.Dense(512, activation="softmax")(x)

outputs = keras.layers.Dense(num_classes, activation="softmax")(x)
net = keras.Model(inputs, outputs)
In [23]:
callback = keras.callbacks.EarlyStopping(monitor='val_loss', patience=10)
In [24]:
lr_schedule = keras.optimizers.schedules.InverseTimeDecay(
    initial_learning_rate = 0.0005,
    decay_steps = 65,
    decay_rate = 0.08)
optimizer = keras.optimizers.Adam(learning_rate=lr_schedule)
net.compile(loss=keras.losses.categorical_crossentropy,
            optimizer=optimizer,
            metrics=['accuracy'])
In [25]:
# Model training
history = net.fit(train_ds_256,
          epochs=80,
          validation_data=val_ds,
            callbacks=[callback])

Epoch 1/80
230/230 [==============================] - 56s 216ms/step - loss: 1.6041 - accuracy: 0.5068 - val_loss: 1.0993 - val_accuracy: 0.6100
Epoch 2/80
230/230 [==============================] - 68s 296ms/step - loss: 1.1423 - accuracy: 0.5954 - val_loss: 0.9347 - val_accuracy: 0.6348
Epoch 3/80
230/230 [==============================] - 111s 481ms/step - loss: 1.0418 - accuracy: 0.6224 - val_loss: 0.7013 - val_accuracy: 0.7294
Epoch 4/80
230/230 [==============================] - 118s 513ms/step - loss: 0.9432 - accuracy: 0.6508 - val_loss: 0.6684 - val_accuracy: 0.7421
Epoch 5/80
230/230 [==============================] - 118s 515ms/step - loss: 0.9024 - accuracy: 0.6660 - val_loss: 0.6577 - val_accuracy: 0.7510
Epoch 6/80
230/230 [==============================] - 119s 516ms/step - loss: 0.8664 - accuracy: 0.6726 - val_loss: 0.6362 - val_accuracy: 0.7622
Epoch 7/80
230/230 [==============================] - 118s 514ms/step - loss: 0.8267 - accuracy: 0.6913 - val_loss: 0.6272 - val_accuracy: 0.7574
Epoch 8/80
230/230 [==============================] - 113s 491ms/step - loss: 0.8182 - accuracy: 0.6963 - val_loss: 0.6504 - val_accuracy: 0.7463
Epoch 9/80
230/230 [==============================] - 110s 477ms/step - loss: 0.8076 - accuracy: 0.6956 - val_loss: 0.6295 - val_accuracy: 0.7625
Epoch 10/80
230/230 [==============================] - 109s 473ms/step - loss: 0.7818 - accuracy: 0.7074 - val_loss: 0.5929 - val_accuracy: 0.7810
Epoch 11/80
230/230 [==============================] - 107s 467ms/step - loss: 0.7640 - accuracy: 0.7117 - val_loss: 0.6208 - val_accuracy: 0.7638
Epoch 12/80
230/230 [==============================] - 108s 471ms/step - loss: 0.7643 - accuracy: 0.7132 - val_loss: 0.5929 - val_accuracy: 0.7765
Epoch 13/80
230/230 [==============================] - 106s 462ms/step - loss: 0.7347 - accuracy: 0.7233 - val_loss: 0.5983 - val_accuracy: 0.7803
Epoch 14/80
230/230 [==============================] - 105s 456ms/step - loss: 0.7364 - accuracy: 0.7239 - val_loss: 0.5549 - val_accuracy: 0.7956
Epoch 15/80
230/230 [==============================] - 105s 457ms/step - loss: 0.7230 - accuracy: 0.7302 - val_loss: 0.5607 - val_accuracy: 0.7950
Epoch 16/80
230/230 [==============================] - 107s 467ms/step - loss: 0.6955 - accuracy: 0.7352 - val_loss: 0.5567 - val_accuracy: 0.7991
Epoch 17/80
230/230 [==============================] - 106s 460ms/step - loss: 0.7070 - accuracy: 0.7351 - val_loss: 0.5535 - val_accuracy: 0.7959
Epoch 18/80
230/230 [==============================] - 105s 456ms/step - loss: 0.7050 - accuracy: 0.7300 - val_loss: 0.5349 - val_accuracy: 0.8039
Epoch 19/80
230/230 [==============================] - 103s 448ms/step - loss: 0.6865 - accuracy: 0.7431 - val_loss: 0.5383 - val_accuracy: 0.8090
Epoch 20/80
230/230 [==============================] - 104s 451ms/step - loss: 0.6795 - accuracy: 0.7400 - val_loss: 0.5268 - val_accuracy: 0.8109
Epoch 21/80
230/230 [==============================] - 107s 467ms/step - loss: 0.6812 - accuracy: 0.7367 - val_loss: 0.5604 - val_accuracy: 0.7934
Epoch 22/80
230/230 [==============================] - 105s 458ms/step - loss: 0.6568 - accuracy: 0.7543 - val_loss: 0.5588 - val_accuracy: 0.7947
Epoch 23/80
230/230 [==============================] - 108s 468ms/step - loss: 0.6432 - accuracy: 0.7529 - val_loss: 0.5541 - val_accuracy: 0.7857
Epoch 24/80
230/230 [==============================] - 107s 465ms/step - loss: 0.6574 - accuracy: 0.7553 - val_loss: 0.5142 - val_accuracy: 0.8150
Epoch 25/80
230/230 [==============================] - 118s 513ms/step - loss: 0.6631 - accuracy: 0.7479 - val_loss: 0.5148 - val_accuracy: 0.8153
Epoch 26/80
230/230 [==============================] - 118s 514ms/step - loss: 0.6404 - accuracy: 0.7536 - val_loss: 0.5091 - val_accuracy: 0.8169
Epoch 27/80
230/230 [==============================] - 118s 514ms/step - loss: 0.6502 - accuracy: 0.7538 - val_loss: 0.5572 - val_accuracy: 0.7911
Epoch 28/80
230/230 [==============================] - 118s 515ms/step - loss: 0.6407 - accuracy: 0.7587 - val_loss: 0.5154 - val_accuracy: 0.8153
Epoch 29/80
230/230 [==============================] - 119s 515ms/step - loss: 0.6318 - accuracy: 0.7539 - val_loss: 0.5168 - val_accuracy: 0.8109
Epoch 30/80
230/230 [==============================] - 119s 515ms/step - loss: 0.6258 - accuracy: 0.7600 - val_loss: 0.5265 - val_accuracy: 0.8083
Epoch 31/80
230/230 [==============================] - 119s 515ms/step - loss: 0.6236 - accuracy: 0.7683 - val_loss: 0.5033 - val_accuracy: 0.8153
Epoch 32/80
230/230 [==============================] - 119s 516ms/step - loss: 0.6162 - accuracy: 0.7663 - val_loss: 0.5001 - val_accuracy: 0.8201
Epoch 33/80
230/230 [==============================] - 119s 516ms/step - loss: 0.6100 - accuracy: 0.7681 - val_loss: 0.5017 - val_accuracy: 0.8192
Epoch 34/80
230/230 [==============================] - 119s 516ms/step - loss: 0.6108 - accuracy: 0.7644 - val_loss: 0.4933 - val_accuracy: 0.8220
Epoch 35/80
230/230 [==============================] - 119s 516ms/step - loss: 0.6154 - accuracy: 0.7681 - val_loss: 0.4927 - val_accuracy: 0.8268
Epoch 36/80
230/230 [==============================] - 119s 516ms/step - loss: 0.6190 - accuracy: 0.7666 - val_loss: 0.4992 - val_accuracy: 0.8255
Epoch 37/80
230/230 [==============================] - 119s 516ms/step - loss: 0.5945 - accuracy: 0.7759 - val_loss: 0.4868 - val_accuracy: 0.8246
Epoch 38/80
230/230 [==============================] - 119s 517ms/step - loss: 0.6008 - accuracy: 0.7722 - val_loss: 0.4893 - val_accuracy: 0.8262
Epoch 39/80
230/230 [==============================] - 120s 519ms/step - loss: 0.6066 - accuracy: 0.7694 - val_loss: 0.4934 - val_accuracy: 0.8255
Epoch 40/80
230/230 [==============================] - 119s 516ms/step - loss: 0.6041 - accuracy: 0.7739 - val_loss: 0.4841 - val_accuracy: 0.8281
Epoch 41/80
230/230 [==============================] - 119s 516ms/step - loss: 0.5981 - accuracy: 0.7677 - val_loss: 0.4804 - val_accuracy: 0.8309
Epoch 42/80
230/230 [==============================] - 119s 516ms/step - loss: 0.6073 - accuracy: 0.7696 - val_loss: 0.4815 - val_accuracy: 0.8281
Epoch 43/80
230/230 [==============================] - 119s 516ms/step - loss: 0.5979 - accuracy: 0.7731 - val_loss: 0.4856 - val_accuracy: 0.8297
Epoch 44/80
230/230 [==============================] - 119s 517ms/step - loss: 0.5800 - accuracy: 0.7802 - val_loss: 0.4882 - val_accuracy: 0.8262
Epoch 45/80
230/230 [==============================] - 119s 516ms/step - loss: 0.5925 - accuracy: 0.7756 - val_loss: 0.4739 - val_accuracy: 0.8313
Epoch 46/80
230/230 [==============================] - 119s 515ms/step - loss: 0.5855 - accuracy: 0.7761 - val_loss: 0.4711 - val_accuracy: 0.8303
Epoch 47/80
230/230 [==============================] - 119s 516ms/step - loss: 0.5814 - accuracy: 0.7790 - val_loss: 0.4785 - val_accuracy: 0.8306
Epoch 48/80
230/230 [==============================] - 119s 518ms/step - loss: 0.5863 - accuracy: 0.7779 - val_loss: 0.4771 - val_accuracy: 0.8287
Epoch 49/80
230/230 [==============================] - 119s 516ms/step - loss: 0.5689 - accuracy: 0.7898 - val_loss: 0.4743 - val_accuracy: 0.8281
Epoch 50/80
230/230 [==============================] - 118s 515ms/step - loss: 0.5885 - accuracy: 0.7759 - val_loss: 0.4901 - val_accuracy: 0.8271
Epoch 51/80
230/230 [==============================] - 118s 514ms/step - loss: 0.5836 - accuracy: 0.7737 - val_loss: 0.4759 - val_accuracy: 0.8303
Epoch 52/80
230/230 [==============================] - 118s 515ms/step - loss: 0.5725 - accuracy: 0.7861 - val_loss: 0.4689 - val_accuracy: 0.8316
Epoch 53/80
230/230 [==============================] - 118s 515ms/step - loss: 0.5855 - accuracy: 0.7827 - val_loss: 0.4786 - val_accuracy: 0.8278
Epoch 54/80
230/230 [==============================] - 119s 519ms/step - loss: 0.5769 - accuracy: 0.7819 - val_loss: 0.4645 - val_accuracy: 0.8332
Epoch 55/80
230/230 [==============================] - 123s 533ms/step - loss: 0.5565 - accuracy: 0.7941 - val_loss: 0.4703 - val_accuracy: 0.8294
Epoch 56/80
230/230 [==============================] - 122s 529ms/step - loss: 0.5632 - accuracy: 0.7873 - val_loss: 0.4996 - val_accuracy: 0.8179
Epoch 57/80
230/230 [==============================] - 121s 527ms/step - loss: 0.5678 - accuracy: 0.7839 - val_loss: 0.4698 - val_accuracy: 0.8325
Epoch 58/80
230/230 [==============================] - 123s 536ms/step - loss: 0.5632 - accuracy: 0.7877 - val_loss: 0.4773 - val_accuracy: 0.8300
Epoch 59/80
230/230 [==============================] - 122s 530ms/step - loss: 0.5626 - accuracy: 0.7864 - val_loss: 0.4686 - val_accuracy: 0.8344
Epoch 60/80
230/230 [==============================] - 121s 524ms/step - loss: 0.5515 - accuracy: 0.7909 - val_loss: 0.4634 - val_accuracy: 0.8392
Epoch 61/80
230/230 [==============================] - 120s 523ms/step - loss: 0.5598 - accuracy: 0.7855 - val_loss: 0.4741 - val_accuracy: 0.8297
Epoch 62/80
230/230 [==============================] - 122s 528ms/step - loss: 0.5647 - accuracy: 0.7881 - val_loss: 0.4608 - val_accuracy: 0.8364
Epoch 63/80
230/230 [==============================] - 121s 523ms/step - loss: 0.5552 - accuracy: 0.7917 - val_loss: 0.4656 - val_accuracy: 0.8341
Epoch 64/80
230/230 [==============================] - 122s 531ms/step - loss: 0.5576 - accuracy: 0.7920 - val_loss: 0.4615 - val_accuracy: 0.8341
Epoch 65/80
230/230 [==============================] - 122s 531ms/step - loss: 0.5563 - accuracy: 0.7926 - val_loss: 0.4753 - val_accuracy: 0.8297
Epoch 66/80
230/230 [==============================] - 123s 535ms/step - loss: 0.5555 - accuracy: 0.7899 - val_loss: 0.4627 - val_accuracy: 0.8348
Epoch 67/80
230/230 [==============================] - 124s 540ms/step - loss: 0.5649 - accuracy: 0.7866 - val_loss: 0.4673 - val_accuracy: 0.8306
Epoch 68/80
230/230 [==============================] - 122s 531ms/step - loss: 0.5572 - accuracy: 0.7905 - val_loss: 0.4634 - val_accuracy: 0.8367
Epoch 69/80
230/230 [==============================] - 119s 517ms/step - loss: 0.5520 - accuracy: 0.7903 - val_loss: 0.4665 - val_accuracy: 0.8287
Epoch 70/80
230/230 [==============================] - 119s 517ms/step - loss: 0.5542 - accuracy: 0.7874 - val_loss: 0.4608 - val_accuracy: 0.8313
Epoch 71/80
230/230 [==============================] - 119s 516ms/step - loss: 0.5486 - accuracy: 0.7929 - val_loss: 0.4679 - val_accuracy: 0.8335
Epoch 72/80
230/230 [==============================] - 119s 519ms/step - loss: 0.5297 - accuracy: 0.8022 - val_loss: 0.4627 - val_accuracy: 0.8325
Epoch 73/80
230/230 [==============================] - 119s 517ms/step - loss: 0.5444 - accuracy: 0.7896 - val_loss: 0.4620 - val_accuracy: 0.8338
Epoch 74/80
230/230 [==============================] - 119s 516ms/step - loss: 0.5504 - accuracy: 0.7920 - val_loss: 0.4603 - val_accuracy: 0.8338
Epoch 75/80
230/230 [==============================] - 119s 516ms/step - loss: 0.5317 - accuracy: 0.8010 - val_loss: 0.4742 - val_accuracy: 0.8262
Epoch 76/80
230/230 [==============================] - 119s 516ms/step - loss: 0.5329 - accuracy: 0.7954 - val_loss: 0.4629 - val_accuracy: 0.8322
Epoch 77/80
230/230 [==============================] - 119s 516ms/step - loss: 0.5353 - accuracy: 0.7974 - val_loss: 0.4688 - val_accuracy: 0.8309
Epoch 78/80
230/230 [==============================] - 119s 516ms/step - loss: 0.5404 - accuracy: 0.7995 - val_loss: 0.4588 - val_accuracy: 0.8348
Epoch 79/80
230/230 [==============================] - 119s 516ms/step - loss: 0.5370 - accuracy: 0.8011 - val_loss: 0.4520 - val_accuracy: 0.8367
Epoch 80/80
230/230 [==============================] - 119s 516ms/step - loss: 0.5424 - accuracy: 0.7926 - val_loss: 0.4679 - val_accuracy: 0.8290
In [26]:
# Visualization of the learning curves

plt.subplot(1, 2, 1)
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.ylim([0.0, 2.0])
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.legend(['train', 'valid'])

plt.subplot(1, 2, 2)
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.ylim([0.5, 1.0])
plt.xlabel('Epoch')
plt.ylabel('Accuracy')
plt.legend(['train', 'valid'])
Out[26]:
<matplotlib.legend.Legend at 0x1c694485ee0>
In [27]:
net.save('models/Segmented2RMS.keras')
In [ ]:
net.evaluate(test_ds)

6/6 [==============================] - 55s 8s/step - loss: 0.5306 - accuracy: 0.8120

[0.5306099057197571, 0.8119891285896301]
In [ ]:
net.evaluate(test_ds_cropped)

21/21 [==============================] - 158s 8s/step - loss: 0.5544 - accuracy: 0.8108

[0.5543912053108215, 0.810800313949585]
In [ ]:
net.evaluate(test_ds_cropped_balanced)

6/6 [==============================] - 43s 7s/step - loss: 0.4482 - accuracy: 0.8470

[0.44818899035453796, 0.8469945192337036]

None of the three accuracy resulted to be higher than the test accuracy we achieved with the model with unsegmented data (84.74%).

Lowering the Dropout rates¶

Since the dropout rates we set for the model with unsegmented data seems to be excessive now, we are going to retry this same architecture on the segmented data, but changing the DropOut rates. In particular, we are going to remove the Dropout Layer from the last 1024 Dense layer.

In [ ]:
# Architecture definition
inputs = keras.Input((None,None,3))

x = inputs

x = keras.layers.RandomCrop(224, 224)(x)
x = keras.layers.RandomFlip(mode='horizontal')(x)
x = keras.layers.RandomContrast(0.25)(x)

x = keras.layers.Conv2D(64, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.MaxPooling2D(3, strides=3, padding="same")(x)
x = keras.layers.Dropout(0.1)(x)

x = keras.layers.Conv2D(128, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.MaxPooling2D(3, strides=3, padding="same")(x)
x = keras.layers.Dropout(0.1)(x)

x = keras.layers.Conv2D(256, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.Dropout(0.1)(x)

x = keras.layers.GlobalMaxPooling2D()(x)

outputs = keras.layers.Dense(1024, activation="softmax")(x)

outputs = keras.layers.Dense(512, activation="softmax")(x)

outputs = keras.layers.Dense(num_classes, activation="softmax")(x)
net = keras.Model(inputs, outputs)
In [ ]:
early_stopping = keras.callbacks.EarlyStopping(monitor='val_loss', patience=10)
In [ ]:
checkpoint_filepath = "/Checkpoint"
checkpoint_callback = tf.keras.callbacks.ModelCheckpoint(
    filepath=checkpoint_filepath,
    monitor='val_accuracy',
    mode='max')
In [ ]:
lr_schedule = keras.optimizers.schedules.InverseTimeDecay(
    initial_learning_rate = 0.0005,
    decay_steps = 65,
    decay_rate = 0.08)
optimizer = keras.optimizers.Adam(learning_rate=lr_schedule)
net.compile(loss=keras.losses.categorical_crossentropy,
            optimizer=optimizer,
            metrics=['accuracy'])
In [ ]:
# Model training
history = net.fit(train_ds_256,
          epochs=80,
          validation_data=val_ds,
            callbacks=[early_stopping, checkpoint_callback])

Epoch 1/80
230/230 [==============================] - 2443s 10s/step - loss: 1.0213 - accuracy: 0.6191 - val_loss: 0.8483 - val_accuracy: 0.6600
Epoch 2/80
230/230 [==============================] - 55s 234ms/step - loss: 0.7569 - accuracy: 0.7001 - val_loss: 0.7210 - val_accuracy: 0.7240
Epoch 3/80
230/230 [==============================] - 54s 233ms/step - loss: 0.6879 - accuracy: 0.7334 - val_loss: 0.7451 - val_accuracy: 0.7131
Epoch 4/80
230/230 [==============================] - 54s 231ms/step - loss: 0.6418 - accuracy: 0.7536 - val_loss: 0.6038 - val_accuracy: 0.7787
Epoch 5/80
230/230 [==============================] - 61s 262ms/step - loss: 0.6056 - accuracy: 0.7685 - val_loss: 0.5804 - val_accuracy: 0.7880
Epoch 6/80
230/230 [==============================] - 54s 232ms/step - loss: 0.5775 - accuracy: 0.7808 - val_loss: 0.5654 - val_accuracy: 0.7921
Epoch 7/80
230/230 [==============================] - 54s 233ms/step - loss: 0.5518 - accuracy: 0.7936 - val_loss: 0.5746 - val_accuracy: 0.7838
Epoch 8/80
230/230 [==============================] - 54s 230ms/step - loss: 0.5334 - accuracy: 0.7992 - val_loss: 0.5435 - val_accuracy: 0.8010
Epoch 9/80
230/230 [==============================] - 56s 238ms/step - loss: 0.5182 - accuracy: 0.8094 - val_loss: 0.5254 - val_accuracy: 0.8077
Epoch 10/80
230/230 [==============================] - 55s 234ms/step - loss: 0.5018 - accuracy: 0.8131 - val_loss: 0.5462 - val_accuracy: 0.7975
Epoch 11/80
230/230 [==============================] - 64s 277ms/step - loss: 0.4989 - accuracy: 0.8117 - val_loss: 0.5082 - val_accuracy: 0.8147
Epoch 12/80
230/230 [==============================] - 55s 232ms/step - loss: 0.4804 - accuracy: 0.8199 - val_loss: 0.5100 - val_accuracy: 0.8115
Epoch 13/80
230/230 [==============================] - 56s 237ms/step - loss: 0.4629 - accuracy: 0.8306 - val_loss: 0.4820 - val_accuracy: 0.8309
Epoch 14/80
230/230 [==============================] - 54s 230ms/step - loss: 0.4645 - accuracy: 0.8295 - val_loss: 0.5031 - val_accuracy: 0.8214
Epoch 15/80
230/230 [==============================] - 54s 232ms/step - loss: 0.4578 - accuracy: 0.8273 - val_loss: 0.4641 - val_accuracy: 0.8278
Epoch 16/80
230/230 [==============================] - 54s 233ms/step - loss: 0.4417 - accuracy: 0.8371 - val_loss: 0.4723 - val_accuracy: 0.8249
Epoch 17/80
230/230 [==============================] - 55s 234ms/step - loss: 0.4313 - accuracy: 0.8441 - val_loss: 0.4518 - val_accuracy: 0.8399
Epoch 18/80
230/230 [==============================] - 54s 229ms/step - loss: 0.4214 - accuracy: 0.8456 - val_loss: 0.4462 - val_accuracy: 0.8437
Epoch 19/80
230/230 [==============================] - 54s 232ms/step - loss: 0.4164 - accuracy: 0.8457 - val_loss: 0.4571 - val_accuracy: 0.8344
Epoch 20/80
230/230 [==============================] - 64s 275ms/step - loss: 0.4128 - accuracy: 0.8480 - val_loss: 0.4536 - val_accuracy: 0.8437
Epoch 21/80
230/230 [==============================] - 54s 228ms/step - loss: 0.4069 - accuracy: 0.8509 - val_loss: 0.4428 - val_accuracy: 0.8472
Epoch 22/80
230/230 [==============================] - 55s 235ms/step - loss: 0.4034 - accuracy: 0.8523 - val_loss: 0.4329 - val_accuracy: 0.8462
Epoch 23/80
230/230 [==============================] - 54s 231ms/step - loss: 0.3960 - accuracy: 0.8596 - val_loss: 0.4359 - val_accuracy: 0.8440
Epoch 24/80
230/230 [==============================] - 54s 232ms/step - loss: 0.3859 - accuracy: 0.8602 - val_loss: 0.4339 - val_accuracy: 0.8450
Epoch 25/80
230/230 [==============================] - 55s 236ms/step - loss: 0.3830 - accuracy: 0.8633 - val_loss: 0.4282 - val_accuracy: 0.8529
Epoch 26/80
230/230 [==============================] - 53s 228ms/step - loss: 0.3842 - accuracy: 0.8632 - val_loss: 0.4330 - val_accuracy: 0.8459
Epoch 27/80
230/230 [==============================] - 54s 231ms/step - loss: 0.3871 - accuracy: 0.8585 - val_loss: 0.4272 - val_accuracy: 0.8548
Epoch 28/80
230/230 [==============================] - 55s 234ms/step - loss: 0.3771 - accuracy: 0.8636 - val_loss: 0.4202 - val_accuracy: 0.8539
Epoch 29/80
230/230 [==============================] - 54s 228ms/step - loss: 0.3686 - accuracy: 0.8670 - val_loss: 0.4194 - val_accuracy: 0.8520
Epoch 30/80
230/230 [==============================] - 54s 233ms/step - loss: 0.3677 - accuracy: 0.8692 - val_loss: 0.4242 - val_accuracy: 0.8532
Epoch 31/80
230/230 [==============================] - 55s 235ms/step - loss: 0.3601 - accuracy: 0.8750 - val_loss: 0.4156 - val_accuracy: 0.8539
Epoch 32/80
230/230 [==============================] - 54s 232ms/step - loss: 0.3586 - accuracy: 0.8723 - val_loss: 0.4142 - val_accuracy: 0.8574
Epoch 33/80
230/230 [==============================] - 55s 236ms/step - loss: 0.3551 - accuracy: 0.8716 - val_loss: 0.4036 - val_accuracy: 0.8583
Epoch 34/80
230/230 [==============================] - 55s 233ms/step - loss: 0.3503 - accuracy: 0.8778 - val_loss: 0.4066 - val_accuracy: 0.8564
Epoch 35/80
230/230 [==============================] - 56s 241ms/step - loss: 0.3546 - accuracy: 0.8760 - val_loss: 0.4143 - val_accuracy: 0.8586
Epoch 36/80
230/230 [==============================] - 64s 277ms/step - loss: 0.3534 - accuracy: 0.8746 - val_loss: 0.4034 - val_accuracy: 0.8548
Epoch 37/80
230/230 [==============================] - 55s 235ms/step - loss: 0.3464 - accuracy: 0.8763 - val_loss: 0.3965 - val_accuracy: 0.8606
Epoch 38/80
230/230 [==============================] - 55s 238ms/step - loss: 0.3457 - accuracy: 0.8774 - val_loss: 0.3990 - val_accuracy: 0.8618
Epoch 39/80
230/230 [==============================] - 67s 286ms/step - loss: 0.3363 - accuracy: 0.8823 - val_loss: 0.4006 - val_accuracy: 0.8602
Epoch 40/80
230/230 [==============================] - 55s 237ms/step - loss: 0.3439 - accuracy: 0.8820 - val_loss: 0.3954 - val_accuracy: 0.8609
Epoch 41/80
230/230 [==============================] - 55s 237ms/step - loss: 0.3421 - accuracy: 0.8771 - val_loss: 0.3945 - val_accuracy: 0.8621
Epoch 42/80
230/230 [==============================] - 56s 237ms/step - loss: 0.3320 - accuracy: 0.8857 - val_loss: 0.3975 - val_accuracy: 0.8628
Epoch 43/80
230/230 [==============================] - 56s 239ms/step - loss: 0.3292 - accuracy: 0.8888 - val_loss: 0.4002 - val_accuracy: 0.8618
Epoch 44/80
230/230 [==============================] - 55s 236ms/step - loss: 0.3361 - accuracy: 0.8855 - val_loss: 0.3931 - val_accuracy: 0.8641
Epoch 45/80
230/230 [==============================] - 65s 279ms/step - loss: 0.3306 - accuracy: 0.8879 - val_loss: 0.3926 - val_accuracy: 0.8647
Epoch 46/80
230/230 [==============================] - 65s 281ms/step - loss: 0.3211 - accuracy: 0.8920 - val_loss: 0.3963 - val_accuracy: 0.8618
Epoch 47/80
230/230 [==============================] - 55s 237ms/step - loss: 0.3278 - accuracy: 0.8872 - val_loss: 0.3970 - val_accuracy: 0.8644
Epoch 48/80
230/230 [==============================] - 55s 236ms/step - loss: 0.3223 - accuracy: 0.8883 - val_loss: 0.3863 - val_accuracy: 0.8672
Epoch 49/80
230/230 [==============================] - 55s 234ms/step - loss: 0.3258 - accuracy: 0.8874 - val_loss: 0.3906 - val_accuracy: 0.8656
Epoch 50/80
230/230 [==============================] - 56s 237ms/step - loss: 0.3166 - accuracy: 0.8917 - val_loss: 0.3929 - val_accuracy: 0.8656
Epoch 51/80
230/230 [==============================] - 57s 240ms/step - loss: 0.3183 - accuracy: 0.8873 - val_loss: 0.3953 - val_accuracy: 0.8631
Epoch 52/80
230/230 [==============================] - 56s 238ms/step - loss: 0.3218 - accuracy: 0.8883 - val_loss: 0.3910 - val_accuracy: 0.8612
Epoch 53/80
230/230 [==============================] - 55s 237ms/step - loss: 0.3184 - accuracy: 0.8895 - val_loss: 0.3859 - val_accuracy: 0.8676
Epoch 54/80
230/230 [==============================] - 56s 239ms/step - loss: 0.3118 - accuracy: 0.8948 - val_loss: 0.3811 - val_accuracy: 0.8663
Epoch 55/80
230/230 [==============================] - 56s 240ms/step - loss: 0.3092 - accuracy: 0.8958 - val_loss: 0.3858 - val_accuracy: 0.8679
Epoch 56/80
230/230 [==============================] - 65s 278ms/step - loss: 0.3084 - accuracy: 0.8929 - val_loss: 0.3877 - val_accuracy: 0.8679
Epoch 57/80
230/230 [==============================] - 56s 240ms/step - loss: 0.3064 - accuracy: 0.8960 - val_loss: 0.3829 - val_accuracy: 0.8666
Epoch 58/80
230/230 [==============================] - 56s 239ms/step - loss: 0.3096 - accuracy: 0.8941 - val_loss: 0.3850 - val_accuracy: 0.8663
Epoch 59/80
230/230 [==============================] - 56s 237ms/step - loss: 0.3048 - accuracy: 0.8932 - val_loss: 0.3829 - val_accuracy: 0.8666
Epoch 60/80
230/230 [==============================] - 55s 237ms/step - loss: 0.2992 - accuracy: 0.8989 - val_loss: 0.3810 - val_accuracy: 0.8621
Epoch 61/80
230/230 [==============================] - 55s 235ms/step - loss: 0.3045 - accuracy: 0.8917 - val_loss: 0.3851 - val_accuracy: 0.8631
Epoch 62/80
230/230 [==============================] - 55s 234ms/step - loss: 0.3091 - accuracy: 0.8950 - val_loss: 0.3736 - val_accuracy: 0.8691
Epoch 63/80
230/230 [==============================] - 55s 236ms/step - loss: 0.3066 - accuracy: 0.8965 - val_loss: 0.3774 - val_accuracy: 0.8663
Epoch 64/80
230/230 [==============================] - 55s 235ms/step - loss: 0.2984 - accuracy: 0.9004 - val_loss: 0.3787 - val_accuracy: 0.8676
Epoch 65/80
230/230 [==============================] - 56s 238ms/step - loss: 0.2922 - accuracy: 0.9011 - val_loss: 0.3764 - val_accuracy: 0.8669
Epoch 66/80
230/230 [==============================] - 57s 245ms/step - loss: 0.2937 - accuracy: 0.9031 - val_loss: 0.3818 - val_accuracy: 0.8663
Epoch 67/80
230/230 [==============================] - 56s 240ms/step - loss: 0.2943 - accuracy: 0.9004 - val_loss: 0.3749 - val_accuracy: 0.8679
Epoch 68/80
230/230 [==============================] - 65s 280ms/step - loss: 0.2994 - accuracy: 0.8958 - val_loss: 0.3784 - val_accuracy: 0.8672
Epoch 69/80
230/230 [==============================] - 66s 284ms/step - loss: 0.2965 - accuracy: 0.8963 - val_loss: 0.3809 - val_accuracy: 0.8704
Epoch 70/80
230/230 [==============================] - 66s 282ms/step - loss: 0.2937 - accuracy: 0.8975 - val_loss: 0.3752 - val_accuracy: 0.8707
Epoch 71/80
230/230 [==============================] - 56s 239ms/step - loss: 0.2846 - accuracy: 0.9042 - val_loss: 0.3768 - val_accuracy: 0.8695
Epoch 72/80
230/230 [==============================] - 57s 241ms/step - loss: 0.2934 - accuracy: 0.8996 - val_loss: 0.3750 - val_accuracy: 0.8720
In [ ]:
# Visualization of the learning curves

plt.subplot(1, 2, 1)
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.ylim([0.0, 2.0])
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.legend(['train', 'valid'])

plt.subplot(1, 2, 2)
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.ylim([0.5, 1.0])
plt.xlabel('Epoch')
plt.ylabel('Accuracy')
plt.legend(['train', 'valid'])

<matplotlib.legend.Legend at 0x7cd4d813b130>
In [ ]:
net.save('models/Segmented3.keras')
In [ ]:
net.evaluate(test_ds)

6/6 [==============================] - 214s 37s/step - loss: 0.5025 - accuracy: 0.8256

[0.5025416612625122, 0.8256130814552307]
In [ ]:
net.evaluate(test_ds_cropped)

21/21 [==============================] - 696s 33s/step - loss: 0.5081 - accuracy: 0.8302

[0.508144736289978, 0.8302253484725952]
In [ ]:
net.evaluate(test_ds_cropped_balanced)

6/6 [==============================] - 165s 27s/step - loss: 0.4214 - accuracy: 0.8484

[0.4213521182537079, 0.8483606576919556]

The accuracy achieved on the test set is notably lower than the accuracy achieved in the validation set, this could be signed of overfitting. The only train accuracy that appears to be higher than the one achieved in the original model (84.74%) is the one achieved in the balanced and segmented test set (84.84%). This could signal better performances on the model if applied to data that is already segmented. Though, the gap is not big.

Balancing the dataset¶

As we said before, the classes in this segmented dataset are unbalanced, we are going to leave dandelion to 1275, and subset every other flower class. Subsetting is not easy, since we have to make sure that each original image is in the subset.

In [ ]:
# Original dimensions of classes
diz = {}
for clas in classes:
    diz[clas] = len(os.listdir(train_path + clas))

diz

{'daisy': 2477,
 'dandelion': 1275,
 'roses': 1504,
 'sunflowers': 2229,
 'tulips': 2986}
In [46]:
subsets = {}
for clas in classes:
    lst = []
    listdir = os.listdir(train_path + clas)
    if clas == 'dandelion':
        subsets[clas] = listdir
        continue
    uniques = []
    for img_path in listdir:
        # Search for the unique images
        unique = re.findall(r'(.*)_\d+.jpg', img_path)[0]
        uniques.append(unique)
    uniques = set(uniques)
    for unique in uniques:
        batch = []
        for img_path in listdir:
            if unique in img_path:
                batch.append(img_path)
        # Prendine uno a caso da batch
        random_element = random.choice(batch)
        # Aggiungilo a lst
        lst.append(random_element)
        # Rimuovilo da listdir
        listdir.remove(random_element)
    # Ora siamo sicuri di avere un alemenento per ciascuna foto, prendiamo gli altri casualmenti da quelli rimasti in listdir
    lst = lst + random.sample(listdir, 1275 - len(lst))
    # Save it in the dictionary
    subsets[clas] = lst
In [47]:
diz = {}
for clas in classes:
    diz[clas] = len(subsets[clas])
diz
Out[47]:
{'daisy': 1275,
 'dandelion': 1275,
 'roses': 1275,
 'sunflowers': 1275,
 'tulips': 1275}
In [50]:
train_balanced_path = 'C:/Users/david/Desktop/DeepLearning/flower_cropped/TrainBalanced/'
# Remove files that are not in the dictionary
for clas in classes:
    for img in os.listdir(train_balanced_path + clas):
        if img not in subsets[clas]:
            os.remove(train_balanced_path + clas + '/' + img)
In [51]:
base_path = 'C:/Users/david/Desktop/DeepLearning/flower_cropped/'

train_ds = keras.preprocessing.image_dataset_from_directory(base_path + 'TrainBalanced',
                                                            image_size=(224,224),
                                                            batch_size=32,
                                                            label_mode='categorical',
                                                            subset='training',
                                                            validation_split = 0.3,
                                                            shuffle=True,
                                                            seed=1)

val_ds = keras.preprocessing.image_dataset_from_directory(base_path + 'TrainBalanced',
                                                            image_size=(224,224),
                                                            batch_size=32,
                                                            label_mode='categorical',
                                                            subset='validation',
                                                            validation_split = 0.3,
                                                            shuffle=True,
                                                            seed=1)

# Data loader with (slightly) larger images
train_ds_256 = keras.preprocessing.image_dataset_from_directory(base_path + 'TrainBalanced',
                                                            image_size=(256,256),
                                                            batch_size=32,
                                                            label_mode='categorical',
                                                            subset='training',
                                                            validation_split = 0.3,
                                                            shuffle=True,
                                                            seed=1)

num_classes = 5

Found 6375 files belonging to 5 classes.
Using 4463 files for training.
Found 6375 files belonging to 5 classes.
Using 1912 files for validation.
Found 6375 files belonging to 5 classes.
Using 4463 files for training.

CNNs with Segmented and Balanced data¶

First Model¶

Testing the performances of the first of the two best models:

In [52]:
# Architecture definition
inputs = keras.Input((None,None,3))

x = inputs

x = keras.layers.RandomCrop(224, 224)(x)
x = keras.layers.RandomFlip(mode='horizontal')(x)
x = keras.layers.RandomContrast(0.25)(x)

x = keras.layers.Conv2D(32, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.MaxPooling2D(3, strides=3, padding="same")(x)

x = keras.layers.Conv2D(64, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.MaxPooling2D(3, strides=3, padding="same")(x)
x = keras.layers.Dropout(0.1)(x)

x = keras.layers.Conv2D(128, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.Dropout(0.2)(x)

x = keras.layers.GlobalMaxPooling2D()(x)

outputs = keras.layers.Dense(num_classes, activation="softmax")(x)
net = keras.Model(inputs, outputs)
In [53]:
callback = keras.callbacks.EarlyStopping(monitor='val_loss', patience=10)
In [54]:
lr_schedule = keras.optimizers.schedules.InverseTimeDecay(
    initial_learning_rate = 0.0005,
    decay_steps = 65,
    decay_rate = 0.08)
optimizer = keras.optimizers.Adam(learning_rate=lr_schedule)
net.compile(loss=keras.losses.categorical_crossentropy,
            optimizer=optimizer,
            metrics=['accuracy'])
In [55]:
# Model training
history = net.fit(train_ds_256,
          epochs=80,
          validation_data=val_ds,
            callbacks=[callback])

Epoch 1/80
140/140 [==============================] - 21s 117ms/step - loss: 1.2473 - accuracy: 0.5326 - val_loss: 1.0856 - val_accuracy: 0.5502
Epoch 2/80
140/140 [==============================] - 15s 107ms/step - loss: 0.8820 - accuracy: 0.6511 - val_loss: 0.8454 - val_accuracy: 0.6606
Epoch 3/80
140/140 [==============================] - 15s 109ms/step - loss: 0.8228 - accuracy: 0.6758 - val_loss: 0.8069 - val_accuracy: 0.6668
Epoch 4/80
140/140 [==============================] - 15s 106ms/step - loss: 0.7680 - accuracy: 0.6998 - val_loss: 0.7246 - val_accuracy: 0.7171
Epoch 5/80
140/140 [==============================] - 15s 107ms/step - loss: 0.7417 - accuracy: 0.7150 - val_loss: 0.7215 - val_accuracy: 0.7265
Epoch 6/80
140/140 [==============================] - 15s 108ms/step - loss: 0.7069 - accuracy: 0.7242 - val_loss: 0.7217 - val_accuracy: 0.7176
Epoch 7/80
140/140 [==============================] - 15s 109ms/step - loss: 0.6682 - accuracy: 0.7464 - val_loss: 0.7310 - val_accuracy: 0.7113
Epoch 8/80
140/140 [==============================] - 15s 107ms/step - loss: 0.6528 - accuracy: 0.7475 - val_loss: 0.6889 - val_accuracy: 0.7395
Epoch 9/80
140/140 [==============================] - 16s 111ms/step - loss: 0.6388 - accuracy: 0.7560 - val_loss: 0.6740 - val_accuracy: 0.7395
Epoch 10/80
140/140 [==============================] - 15s 107ms/step - loss: 0.6217 - accuracy: 0.7643 - val_loss: 0.6499 - val_accuracy: 0.7563
Epoch 11/80
140/140 [==============================] - 17s 118ms/step - loss: 0.6197 - accuracy: 0.7683 - val_loss: 0.6418 - val_accuracy: 0.7641
Epoch 12/80
140/140 [==============================] - 19s 136ms/step - loss: 0.6162 - accuracy: 0.7659 - val_loss: 0.6204 - val_accuracy: 0.7845
Epoch 13/80
140/140 [==============================] - 20s 146ms/step - loss: 0.5863 - accuracy: 0.7777 - val_loss: 0.6329 - val_accuracy: 0.7683
Epoch 14/80
140/140 [==============================] - 22s 157ms/step - loss: 0.5957 - accuracy: 0.7688 - val_loss: 0.6246 - val_accuracy: 0.7657
Epoch 15/80
140/140 [==============================] - 23s 165ms/step - loss: 0.5853 - accuracy: 0.7737 - val_loss: 0.6050 - val_accuracy: 0.7793
Epoch 16/80
140/140 [==============================] - 26s 187ms/step - loss: 0.5674 - accuracy: 0.7883 - val_loss: 0.6299 - val_accuracy: 0.7641
Epoch 17/80
140/140 [==============================] - 27s 190ms/step - loss: 0.5560 - accuracy: 0.7968 - val_loss: 0.6052 - val_accuracy: 0.7756
Epoch 18/80
140/140 [==============================] - 28s 199ms/step - loss: 0.5577 - accuracy: 0.7887 - val_loss: 0.5946 - val_accuracy: 0.7803
Epoch 19/80
140/140 [==============================] - 30s 213ms/step - loss: 0.5493 - accuracy: 0.7948 - val_loss: 0.5928 - val_accuracy: 0.7819
Epoch 20/80
140/140 [==============================] - 30s 215ms/step - loss: 0.5356 - accuracy: 0.8039 - val_loss: 0.5859 - val_accuracy: 0.7871
Epoch 21/80
140/140 [==============================] - 29s 207ms/step - loss: 0.5387 - accuracy: 0.7965 - val_loss: 0.6131 - val_accuracy: 0.7730
Epoch 22/80
140/140 [==============================] - 31s 225ms/step - loss: 0.5266 - accuracy: 0.8082 - val_loss: 0.5709 - val_accuracy: 0.8007
Epoch 23/80
140/140 [==============================] - 30s 218ms/step - loss: 0.5322 - accuracy: 0.8066 - val_loss: 0.5909 - val_accuracy: 0.7887
Epoch 24/80
140/140 [==============================] - 32s 231ms/step - loss: 0.5252 - accuracy: 0.7997 - val_loss: 0.5752 - val_accuracy: 0.7903
Epoch 25/80
140/140 [==============================] - 33s 238ms/step - loss: 0.5207 - accuracy: 0.8111 - val_loss: 0.5753 - val_accuracy: 0.7924
Epoch 26/80
140/140 [==============================] - 34s 247ms/step - loss: 0.5274 - accuracy: 0.8004 - val_loss: 0.5750 - val_accuracy: 0.7861
Epoch 27/80
140/140 [==============================] - 34s 244ms/step - loss: 0.5046 - accuracy: 0.8136 - val_loss: 0.5592 - val_accuracy: 0.7976
Epoch 28/80
140/140 [==============================] - 35s 246ms/step - loss: 0.5087 - accuracy: 0.8151 - val_loss: 0.5679 - val_accuracy: 0.7945
Epoch 29/80
140/140 [==============================] - 35s 249ms/step - loss: 0.4979 - accuracy: 0.8156 - val_loss: 0.5527 - val_accuracy: 0.7997
Epoch 30/80
140/140 [==============================] - 35s 247ms/step - loss: 0.4941 - accuracy: 0.8205 - val_loss: 0.5591 - val_accuracy: 0.7945
Epoch 31/80
140/140 [==============================] - 34s 246ms/step - loss: 0.4890 - accuracy: 0.8219 - val_loss: 0.5582 - val_accuracy: 0.7960
Epoch 32/80
140/140 [==============================] - 35s 249ms/step - loss: 0.4902 - accuracy: 0.8196 - val_loss: 0.5575 - val_accuracy: 0.7986
Epoch 33/80
140/140 [==============================] - 36s 253ms/step - loss: 0.4903 - accuracy: 0.8201 - val_loss: 0.5571 - val_accuracy: 0.7939
Epoch 34/80
140/140 [==============================] - 35s 250ms/step - loss: 0.4706 - accuracy: 0.8286 - val_loss: 0.5544 - val_accuracy: 0.8007
Epoch 35/80
140/140 [==============================] - 35s 249ms/step - loss: 0.4836 - accuracy: 0.8230 - val_loss: 0.5398 - val_accuracy: 0.8070
Epoch 36/80
140/140 [==============================] - 35s 250ms/step - loss: 0.4824 - accuracy: 0.8263 - val_loss: 0.5833 - val_accuracy: 0.7856
Epoch 37/80
140/140 [==============================] - 35s 250ms/step - loss: 0.4672 - accuracy: 0.8241 - val_loss: 0.5485 - val_accuracy: 0.8002
Epoch 38/80
140/140 [==============================] - 35s 249ms/step - loss: 0.4781 - accuracy: 0.8259 - val_loss: 0.5311 - val_accuracy: 0.8143
Epoch 39/80
140/140 [==============================] - 35s 249ms/step - loss: 0.4722 - accuracy: 0.8288 - val_loss: 0.5307 - val_accuracy: 0.8180
Epoch 40/80
140/140 [==============================] - 35s 247ms/step - loss: 0.4668 - accuracy: 0.8286 - val_loss: 0.5376 - val_accuracy: 0.8075
Epoch 41/80
140/140 [==============================] - 35s 248ms/step - loss: 0.4708 - accuracy: 0.8243 - val_loss: 0.5433 - val_accuracy: 0.8033
Epoch 42/80
140/140 [==============================] - 35s 248ms/step - loss: 0.4632 - accuracy: 0.8322 - val_loss: 0.5352 - val_accuracy: 0.8049
Epoch 43/80
140/140 [==============================] - 35s 249ms/step - loss: 0.4626 - accuracy: 0.8281 - val_loss: 0.5432 - val_accuracy: 0.8018
Epoch 44/80
140/140 [==============================] - 35s 249ms/step - loss: 0.4505 - accuracy: 0.8367 - val_loss: 0.5401 - val_accuracy: 0.7997
Epoch 45/80
140/140 [==============================] - 35s 250ms/step - loss: 0.4517 - accuracy: 0.8342 - val_loss: 0.5316 - val_accuracy: 0.8075
Epoch 46/80
140/140 [==============================] - 35s 249ms/step - loss: 0.4584 - accuracy: 0.8373 - val_loss: 0.5352 - val_accuracy: 0.8065
Epoch 47/80
140/140 [==============================] - 35s 250ms/step - loss: 0.4510 - accuracy: 0.8418 - val_loss: 0.5389 - val_accuracy: 0.8007
Epoch 48/80
140/140 [==============================] - 35s 249ms/step - loss: 0.4500 - accuracy: 0.8328 - val_loss: 0.5227 - val_accuracy: 0.8133
Epoch 49/80
140/140 [==============================] - 35s 247ms/step - loss: 0.4443 - accuracy: 0.8371 - val_loss: 0.5237 - val_accuracy: 0.8143
Epoch 50/80
140/140 [==============================] - 35s 246ms/step - loss: 0.4430 - accuracy: 0.8387 - val_loss: 0.5316 - val_accuracy: 0.8081
Epoch 51/80
140/140 [==============================] - 35s 246ms/step - loss: 0.4400 - accuracy: 0.8391 - val_loss: 0.5249 - val_accuracy: 0.8091
Epoch 52/80
140/140 [==============================] - 35s 248ms/step - loss: 0.4346 - accuracy: 0.8423 - val_loss: 0.5230 - val_accuracy: 0.8138
Epoch 53/80
140/140 [==============================] - 35s 250ms/step - loss: 0.4465 - accuracy: 0.8369 - val_loss: 0.5289 - val_accuracy: 0.8081
Epoch 54/80
140/140 [==============================] - 35s 249ms/step - loss: 0.4337 - accuracy: 0.8402 - val_loss: 0.5231 - val_accuracy: 0.8075
Epoch 55/80
140/140 [==============================] - 35s 249ms/step - loss: 0.4175 - accuracy: 0.8481 - val_loss: 0.5398 - val_accuracy: 0.8060
Epoch 56/80
140/140 [==============================] - 35s 249ms/step - loss: 0.4301 - accuracy: 0.8465 - val_loss: 0.5253 - val_accuracy: 0.8096
Epoch 57/80
140/140 [==============================] - 35s 250ms/step - loss: 0.4246 - accuracy: 0.8526 - val_loss: 0.5297 - val_accuracy: 0.8096
Epoch 58/80
140/140 [==============================] - 35s 248ms/step - loss: 0.4330 - accuracy: 0.8427 - val_loss: 0.5247 - val_accuracy: 0.8101
In [56]:
# Visualization of the learning curves

plt.subplot(1, 2, 1)
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.ylim([0.0, 2.0])
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.legend(['train', 'valid'])

plt.subplot(1, 2, 2)
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.ylim([0.5, 1.0])
plt.xlabel('Epoch')
plt.ylabel('Accuracy')
plt.legend(['train', 'valid'])
Out[56]:
<matplotlib.legend.Legend at 0x2b080699460>
In [57]:
net.save('models/BalancedSegmentv1.keras')
In [5]:
net = keras.models.load_model("models/BalancedSegmentv1.keras")
net.evaluate(test_ds)
net.evaluate(test_ds_cropped)
net.evaluate(test_ds_cropped_balanced)

6/6 [==============================] - 9s 531ms/step - loss: 0.5800 - accuracy: 0.8038
21/21 [==============================] - 4s 187ms/step - loss: 0.6111 - accuracy: 0.7848
6/6 [==============================] - 3s 594ms/step - loss: 0.4968 - accuracy: 0.8279
Out[5]:
[0.49677494168281555, 0.8278688788414001]

None of the three accuracy resulted to be higher than the test accuracy we achieved with the model with unsegmented data (83.65%). Also, the balancing of the dataset did not seem to increase the accuracy either, since the accuracy on all three of the test datasets decreased.

Second Model¶

Testing the performances of the second of the two best models:

In [58]:
# Architecture definition
inputs = keras.Input((None,None,3))

x = inputs

x = keras.layers.RandomCrop(224, 224)(x)
x = keras.layers.RandomFlip(mode='horizontal')(x)
x = keras.layers.RandomContrast(0.25)(x)

x = keras.layers.Conv2D(64, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.MaxPooling2D(3, strides=3, padding="same")(x)
x = keras.layers.Dropout(0.1)(x)

x = keras.layers.Conv2D(128, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.MaxPooling2D(3, strides=3, padding="same")(x)
x = keras.layers.Dropout(0.1)(x)

x = keras.layers.Conv2D(256, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.Dropout(0.1)(x)

x = keras.layers.GlobalMaxPooling2D()(x)

outputs = keras.layers.Dense(1024, activation="softmax")(x)
x = keras.layers.Dropout(0.2)(x)

outputs = keras.layers.Dense(512, activation="softmax")(x)

outputs = keras.layers.Dense(num_classes, activation="softmax")(x)
net = keras.Model(inputs, outputs)
In [59]:
callback = keras.callbacks.EarlyStopping(monitor='val_loss', patience=10)
In [60]:
lr_schedule = keras.optimizers.schedules.InverseTimeDecay(
    initial_learning_rate = 0.0005,
    decay_steps = 65,
    decay_rate = 0.08)
optimizer = keras.optimizers.Adam(learning_rate=lr_schedule)
net.compile(loss=keras.losses.categorical_crossentropy,
            optimizer=optimizer,
            metrics=['accuracy'])
In [61]:
# Model training
history = net.fit(train_ds_256,
          epochs=80,
          validation_data=val_ds,
            callbacks=[callback])

Epoch 1/80
140/140 [==============================] - 90s 641ms/step - loss: 1.7278 - accuracy: 0.4613 - val_loss: 1.2653 - val_accuracy: 0.5335
Epoch 2/80
140/140 [==============================] - 75s 538ms/step - loss: 1.2755 - accuracy: 0.5548 - val_loss: 0.9264 - val_accuracy: 0.6245
Epoch 3/80
140/140 [==============================] - 76s 539ms/step - loss: 1.1573 - accuracy: 0.5772 - val_loss: 0.8170 - val_accuracy: 0.6768
Epoch 4/80
140/140 [==============================] - 75s 539ms/step - loss: 1.0786 - accuracy: 0.6034 - val_loss: 0.7736 - val_accuracy: 0.6967
Epoch 5/80
140/140 [==============================] - 75s 539ms/step - loss: 1.0173 - accuracy: 0.6278 - val_loss: 0.7631 - val_accuracy: 0.6987
Epoch 6/80
140/140 [==============================] - 75s 538ms/step - loss: 0.9650 - accuracy: 0.6366 - val_loss: 0.7770 - val_accuracy: 0.6878
Epoch 7/80
140/140 [==============================] - 75s 539ms/step - loss: 0.9596 - accuracy: 0.6361 - val_loss: 0.7392 - val_accuracy: 0.7212
Epoch 8/80
140/140 [==============================] - 75s 537ms/step - loss: 0.9110 - accuracy: 0.6596 - val_loss: 0.7036 - val_accuracy: 0.7306
Epoch 9/80
140/140 [==============================] - 75s 538ms/step - loss: 0.8890 - accuracy: 0.6628 - val_loss: 0.6969 - val_accuracy: 0.7442
Epoch 10/80
140/140 [==============================] - 75s 538ms/step - loss: 0.8666 - accuracy: 0.6711 - val_loss: 0.6588 - val_accuracy: 0.7516
Epoch 11/80
140/140 [==============================] - 75s 539ms/step - loss: 0.8601 - accuracy: 0.6836 - val_loss: 0.6706 - val_accuracy: 0.7432
Epoch 12/80
140/140 [==============================] - 75s 538ms/step - loss: 0.8458 - accuracy: 0.6717 - val_loss: 0.6459 - val_accuracy: 0.7531
Epoch 13/80
140/140 [==============================] - 75s 538ms/step - loss: 0.8221 - accuracy: 0.6892 - val_loss: 0.6402 - val_accuracy: 0.7568
Epoch 14/80
140/140 [==============================] - 75s 538ms/step - loss: 0.8132 - accuracy: 0.6906 - val_loss: 0.6231 - val_accuracy: 0.7652
Epoch 15/80
140/140 [==============================] - 75s 539ms/step - loss: 0.7865 - accuracy: 0.6944 - val_loss: 0.6335 - val_accuracy: 0.7636
Epoch 16/80
140/140 [==============================] - 76s 539ms/step - loss: 0.7923 - accuracy: 0.7031 - val_loss: 0.6196 - val_accuracy: 0.7688
Epoch 17/80
140/140 [==============================] - 75s 538ms/step - loss: 0.7765 - accuracy: 0.6944 - val_loss: 0.6088 - val_accuracy: 0.7720
Epoch 18/80
140/140 [==============================] - 75s 538ms/step - loss: 0.7583 - accuracy: 0.7105 - val_loss: 0.5969 - val_accuracy: 0.7788
Epoch 19/80
140/140 [==============================] - 75s 539ms/step - loss: 0.7458 - accuracy: 0.7170 - val_loss: 0.6255 - val_accuracy: 0.7542
Epoch 20/80
140/140 [==============================] - 75s 536ms/step - loss: 0.7293 - accuracy: 0.7244 - val_loss: 0.5878 - val_accuracy: 0.7819
Epoch 21/80
140/140 [==============================] - 75s 538ms/step - loss: 0.7479 - accuracy: 0.7119 - val_loss: 0.6028 - val_accuracy: 0.7641
Epoch 22/80
140/140 [==============================] - 76s 539ms/step - loss: 0.7432 - accuracy: 0.7177 - val_loss: 0.5969 - val_accuracy: 0.7793
Epoch 23/80
140/140 [==============================] - 75s 539ms/step - loss: 0.7283 - accuracy: 0.7206 - val_loss: 0.5793 - val_accuracy: 0.7882
Epoch 24/80
140/140 [==============================] - 76s 540ms/step - loss: 0.7182 - accuracy: 0.7246 - val_loss: 0.5805 - val_accuracy: 0.7892
Epoch 25/80
140/140 [==============================] - 75s 538ms/step - loss: 0.7285 - accuracy: 0.7231 - val_loss: 0.6001 - val_accuracy: 0.7767
Epoch 26/80
140/140 [==============================] - 75s 539ms/step - loss: 0.7201 - accuracy: 0.7255 - val_loss: 0.5754 - val_accuracy: 0.7882
Epoch 27/80
140/140 [==============================] - 75s 538ms/step - loss: 0.7016 - accuracy: 0.7320 - val_loss: 0.5711 - val_accuracy: 0.7924
Epoch 28/80
140/140 [==============================] - 75s 538ms/step - loss: 0.7142 - accuracy: 0.7240 - val_loss: 0.5714 - val_accuracy: 0.7908
Epoch 29/80
140/140 [==============================] - 75s 538ms/step - loss: 0.6914 - accuracy: 0.7367 - val_loss: 0.5645 - val_accuracy: 0.8013
Epoch 30/80
140/140 [==============================] - 75s 537ms/step - loss: 0.6865 - accuracy: 0.7372 - val_loss: 0.5685 - val_accuracy: 0.7986
Epoch 31/80
140/140 [==============================] - 76s 540ms/step - loss: 0.6951 - accuracy: 0.7309 - val_loss: 0.5738 - val_accuracy: 0.7918
Epoch 32/80
140/140 [==============================] - 75s 537ms/step - loss: 0.6711 - accuracy: 0.7421 - val_loss: 0.5701 - val_accuracy: 0.7908
Epoch 33/80
140/140 [==============================] - 76s 540ms/step - loss: 0.6861 - accuracy: 0.7378 - val_loss: 0.5542 - val_accuracy: 0.8060
Epoch 34/80
140/140 [==============================] - 75s 539ms/step - loss: 0.6760 - accuracy: 0.7417 - val_loss: 0.5545 - val_accuracy: 0.7992
Epoch 35/80
140/140 [==============================] - 75s 539ms/step - loss: 0.6761 - accuracy: 0.7477 - val_loss: 0.5520 - val_accuracy: 0.7997
Epoch 36/80
140/140 [==============================] - 75s 538ms/step - loss: 0.6519 - accuracy: 0.7533 - val_loss: 0.5486 - val_accuracy: 0.8117
Epoch 37/80
140/140 [==============================] - 75s 538ms/step - loss: 0.6613 - accuracy: 0.7365 - val_loss: 0.5604 - val_accuracy: 0.7960
Epoch 38/80
140/140 [==============================] - 75s 538ms/step - loss: 0.6465 - accuracy: 0.7546 - val_loss: 0.5815 - val_accuracy: 0.7688
Epoch 39/80
140/140 [==============================] - 75s 538ms/step - loss: 0.6497 - accuracy: 0.7600 - val_loss: 0.5436 - val_accuracy: 0.8060
Epoch 40/80
140/140 [==============================] - 76s 540ms/step - loss: 0.6622 - accuracy: 0.7428 - val_loss: 0.5481 - val_accuracy: 0.8013
Epoch 41/80
140/140 [==============================] - 76s 540ms/step - loss: 0.6389 - accuracy: 0.7555 - val_loss: 0.5436 - val_accuracy: 0.8060
Epoch 42/80
140/140 [==============================] - 75s 539ms/step - loss: 0.6458 - accuracy: 0.7522 - val_loss: 0.5505 - val_accuracy: 0.8044
Epoch 43/80
140/140 [==============================] - 75s 538ms/step - loss: 0.6440 - accuracy: 0.7531 - val_loss: 0.5482 - val_accuracy: 0.8023
Epoch 44/80
140/140 [==============================] - 75s 536ms/step - loss: 0.6437 - accuracy: 0.7490 - val_loss: 0.5409 - val_accuracy: 0.8049
Epoch 45/80
140/140 [==============================] - 75s 537ms/step - loss: 0.6391 - accuracy: 0.7582 - val_loss: 0.5435 - val_accuracy: 0.8070
Epoch 46/80
140/140 [==============================] - 76s 540ms/step - loss: 0.6191 - accuracy: 0.7571 - val_loss: 0.5336 - val_accuracy: 0.8075
Epoch 47/80
140/140 [==============================] - 75s 539ms/step - loss: 0.6371 - accuracy: 0.7582 - val_loss: 0.5327 - val_accuracy: 0.8122
Epoch 48/80
140/140 [==============================] - 75s 539ms/step - loss: 0.6399 - accuracy: 0.7544 - val_loss: 0.5356 - val_accuracy: 0.8096
Epoch 49/80
140/140 [==============================] - 75s 539ms/step - loss: 0.6301 - accuracy: 0.7598 - val_loss: 0.5462 - val_accuracy: 0.7992
Epoch 50/80
140/140 [==============================] - 75s 539ms/step - loss: 0.6273 - accuracy: 0.7580 - val_loss: 0.5275 - val_accuracy: 0.8180
Epoch 51/80
140/140 [==============================] - 75s 538ms/step - loss: 0.6334 - accuracy: 0.7582 - val_loss: 0.5384 - val_accuracy: 0.8023
Epoch 52/80
140/140 [==============================] - 75s 536ms/step - loss: 0.6291 - accuracy: 0.7614 - val_loss: 0.5494 - val_accuracy: 0.8033
Epoch 53/80
140/140 [==============================] - 75s 537ms/step - loss: 0.6279 - accuracy: 0.7571 - val_loss: 0.5318 - val_accuracy: 0.8128
Epoch 54/80
140/140 [==============================] - 76s 539ms/step - loss: 0.6192 - accuracy: 0.7638 - val_loss: 0.5276 - val_accuracy: 0.8112
Epoch 55/80
140/140 [==============================] - 75s 538ms/step - loss: 0.6010 - accuracy: 0.7676 - val_loss: 0.5367 - val_accuracy: 0.8013
Epoch 56/80
140/140 [==============================] - 75s 536ms/step - loss: 0.6222 - accuracy: 0.7627 - val_loss: 0.5242 - val_accuracy: 0.8143
Epoch 57/80
140/140 [==============================] - 75s 536ms/step - loss: 0.6012 - accuracy: 0.7667 - val_loss: 0.5250 - val_accuracy: 0.8149
Epoch 58/80
140/140 [==============================] - 75s 536ms/step - loss: 0.6040 - accuracy: 0.7694 - val_loss: 0.5273 - val_accuracy: 0.8159
Epoch 59/80
140/140 [==============================] - 75s 537ms/step - loss: 0.5878 - accuracy: 0.7746 - val_loss: 0.5214 - val_accuracy: 0.8149
Epoch 60/80
140/140 [==============================] - 76s 540ms/step - loss: 0.6040 - accuracy: 0.7692 - val_loss: 0.5295 - val_accuracy: 0.8096
Epoch 61/80
140/140 [==============================] - 76s 544ms/step - loss: 0.5974 - accuracy: 0.7661 - val_loss: 0.5164 - val_accuracy: 0.8196
Epoch 62/80
140/140 [==============================] - 76s 539ms/step - loss: 0.5932 - accuracy: 0.7724 - val_loss: 0.5133 - val_accuracy: 0.8164
Epoch 63/80
140/140 [==============================] - 75s 536ms/step - loss: 0.6027 - accuracy: 0.7744 - val_loss: 0.5145 - val_accuracy: 0.8164
Epoch 64/80
140/140 [==============================] - 75s 536ms/step - loss: 0.6014 - accuracy: 0.7694 - val_loss: 0.5131 - val_accuracy: 0.8154
Epoch 65/80
140/140 [==============================] - 75s 536ms/step - loss: 0.5940 - accuracy: 0.7656 - val_loss: 0.5167 - val_accuracy: 0.8206
Epoch 66/80
140/140 [==============================] - 75s 536ms/step - loss: 0.5947 - accuracy: 0.7688 - val_loss: 0.5224 - val_accuracy: 0.8154
Epoch 67/80
140/140 [==============================] - 75s 536ms/step - loss: 0.5984 - accuracy: 0.7739 - val_loss: 0.5115 - val_accuracy: 0.8196
Epoch 68/80
140/140 [==============================] - 75s 536ms/step - loss: 0.6151 - accuracy: 0.7681 - val_loss: 0.5171 - val_accuracy: 0.8138
Epoch 69/80
140/140 [==============================] - 75s 536ms/step - loss: 0.5884 - accuracy: 0.7730 - val_loss: 0.5230 - val_accuracy: 0.8133
Epoch 70/80
140/140 [==============================] - 75s 536ms/step - loss: 0.5795 - accuracy: 0.7782 - val_loss: 0.5145 - val_accuracy: 0.8133
Epoch 71/80
140/140 [==============================] - 75s 536ms/step - loss: 0.5954 - accuracy: 0.7717 - val_loss: 0.5092 - val_accuracy: 0.8227
Epoch 72/80
140/140 [==============================] - 75s 536ms/step - loss: 0.5931 - accuracy: 0.7712 - val_loss: 0.5077 - val_accuracy: 0.8206
Epoch 73/80
140/140 [==============================] - 75s 536ms/step - loss: 0.5912 - accuracy: 0.7831 - val_loss: 0.5080 - val_accuracy: 0.8196
Epoch 74/80
140/140 [==============================] - 75s 536ms/step - loss: 0.5912 - accuracy: 0.7739 - val_loss: 0.5093 - val_accuracy: 0.8201
Epoch 75/80
140/140 [==============================] - 75s 536ms/step - loss: 0.5838 - accuracy: 0.7730 - val_loss: 0.5137 - val_accuracy: 0.8169
Epoch 76/80
140/140 [==============================] - 75s 536ms/step - loss: 0.5849 - accuracy: 0.7813 - val_loss: 0.5097 - val_accuracy: 0.8201
Epoch 77/80
140/140 [==============================] - 75s 536ms/step - loss: 0.5692 - accuracy: 0.7818 - val_loss: 0.5087 - val_accuracy: 0.8169
Epoch 78/80
140/140 [==============================] - 75s 536ms/step - loss: 0.5733 - accuracy: 0.7793 - val_loss: 0.5071 - val_accuracy: 0.8190
Epoch 79/80
140/140 [==============================] - 75s 536ms/step - loss: 0.5783 - accuracy: 0.7824 - val_loss: 0.5133 - val_accuracy: 0.8169
Epoch 80/80
140/140 [==============================] - 76s 540ms/step - loss: 0.5673 - accuracy: 0.7820 - val_loss: 0.5008 - val_accuracy: 0.8290
In [62]:
# Visualization of the learning curves

plt.subplot(1, 2, 1)
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.ylim([0.0, 2.0])
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.legend(['train', 'valid'])

plt.subplot(1, 2, 2)
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.ylim([0.5, 1.0])
plt.xlabel('Epoch')
plt.ylabel('Accuracy')
plt.legend(['train', 'valid'])
Out[62]:
<matplotlib.legend.Legend at 0x2b10ed0e7f0>
In [63]:
net.save('models/BalancedSegmented2.keras')
In [ ]:
net.evaluate(test_ds)

6/6 [==============================] - 78s 12s/step - loss: 0.5201 - accuracy: 0.8106

[0.5200757384300232, 0.8106266856193542]
In [ ]:
net.evaluate(test_ds_cropped)

21/21 [==============================] - 147s 7s/step - loss: 0.5853 - accuracy: 0.8046

[0.5853006839752197, 0.8045843243598938]
In [ ]:
net.evaluate(test_ds_cropped_balanced)

6/6 [==============================] - 42s 7s/step - loss: 0.4550 - accuracy: 0.8320

[0.4550371766090393, 0.8319672346115112]

None of the three accuracy resulted to be higher than the test accuracy we achieved with the same model applied to unsegmented data (84.74%). Also, the balancing of the dataset did not seem to increase the performance of the model, since the performances decreased on all the test sets.

Lowering the Dropout rates¶

As we did with the unbalanced data, since the dropout rates we set for the model with unsegmented data seems to be excessive now, we are going to retry this same architecture on the segmented and balanced data data, but changing the DropOut rates. In particular, we are going to remove the Dropout Layer from the last 1024 Dense layer.

In [ ]:
# Architecture definition
inputs = keras.Input((None,None,3))

x = inputs

x = keras.layers.RandomCrop(224, 224)(x)
x = keras.layers.RandomFlip(mode='horizontal')(x)
x = keras.layers.RandomContrast(0.25)(x)

x = keras.layers.Conv2D(64, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.MaxPooling2D(3, strides=3, padding="same")(x)
x = keras.layers.Dropout(0.1)(x)

x = keras.layers.Conv2D(128, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.MaxPooling2D(3, strides=3, padding="same")(x)
x = keras.layers.Dropout(0.1)(x)

x = keras.layers.Conv2D(256, 3, padding="same")(x)
x = keras.layers.BatchNormalization()(x)
x = keras.layers.Activation("relu")(x)
x = keras.layers.Dropout(0.1)(x)

x = keras.layers.GlobalMaxPooling2D()(x)

outputs = keras.layers.Dense(1024, activation="softmax")(x)

outputs = keras.layers.Dense(512, activation="softmax")(x)

outputs = keras.layers.Dense(num_classes, activation="softmax")(x)
net = keras.Model(inputs, outputs)
In [ ]:
early_stopping = keras.callbacks.EarlyStopping(monitor='val_loss', patience=10)
In [ ]:
checkpoint_filepath = "/content/gdrive/MyDrive/Deep Learning Project/Checkpoint3"
checkpoint_callback = tf.keras.callbacks.ModelCheckpoint(
    filepath=checkpoint_filepath,
    monitor='val_accuracy',
    mode='max')
In [ ]:
lr_schedule = keras.optimizers.schedules.InverseTimeDecay(
    initial_learning_rate = 0.0005,
    decay_steps = 65,
    decay_rate = 0.08)
optimizer = keras.optimizers.Adam(learning_rate=lr_schedule)
net.compile(loss=keras.losses.categorical_crossentropy,
            optimizer=optimizer,
            metrics=['accuracy'])
In [ ]:
# Model training
history = net.fit(train_ds_256,
          epochs=80,
          validation_data=val_ds,
            callbacks=[early_stopping, checkpoint_callback])

Epoch 1/80
140/140 [==============================] - 2457s 17s/step - loss: 1.3198 - accuracy: 0.5337 - val_loss: 0.9736 - val_accuracy: 0.5816
Epoch 2/80
140/140 [==============================] - 35s 247ms/step - loss: 0.8740 - accuracy: 0.6561 - val_loss: 0.7808 - val_accuracy: 0.7040
Epoch 3/80
140/140 [==============================] - 35s 243ms/step - loss: 0.8116 - accuracy: 0.6812 - val_loss: 0.7205 - val_accuracy: 0.7385
Epoch 4/80
140/140 [==============================] - 35s 242ms/step - loss: 0.7513 - accuracy: 0.7022 - val_loss: 0.8024 - val_accuracy: 0.6888
Epoch 5/80
140/140 [==============================] - 36s 247ms/step - loss: 0.7038 - accuracy: 0.7213 - val_loss: 0.7024 - val_accuracy: 0.7327
Epoch 6/80
140/140 [==============================] - 34s 239ms/step - loss: 0.6835 - accuracy: 0.7363 - val_loss: 0.6768 - val_accuracy: 0.7453
Epoch 7/80
140/140 [==============================] - 34s 238ms/step - loss: 0.6472 - accuracy: 0.7479 - val_loss: 0.7409 - val_accuracy: 0.7055
Epoch 8/80
140/140 [==============================] - 36s 250ms/step - loss: 0.6269 - accuracy: 0.7582 - val_loss: 0.6507 - val_accuracy: 0.7547
Epoch 9/80
140/140 [==============================] - 39s 270ms/step - loss: 0.5944 - accuracy: 0.7735 - val_loss: 0.6200 - val_accuracy: 0.7720
Epoch 10/80
140/140 [==============================] - 35s 240ms/step - loss: 0.5818 - accuracy: 0.7797 - val_loss: 0.6247 - val_accuracy: 0.7542
Epoch 11/80
140/140 [==============================] - 38s 268ms/step - loss: 0.5774 - accuracy: 0.7847 - val_loss: 0.5907 - val_accuracy: 0.7777
Epoch 12/80
140/140 [==============================] - 38s 269ms/step - loss: 0.5567 - accuracy: 0.7941 - val_loss: 0.5902 - val_accuracy: 0.7835
Epoch 13/80
140/140 [==============================] - 35s 241ms/step - loss: 0.5406 - accuracy: 0.7952 - val_loss: 0.5719 - val_accuracy: 0.7955
Epoch 14/80
140/140 [==============================] - 35s 243ms/step - loss: 0.5269 - accuracy: 0.8082 - val_loss: 0.5745 - val_accuracy: 0.7856
Epoch 15/80
140/140 [==============================] - 34s 234ms/step - loss: 0.5236 - accuracy: 0.8006 - val_loss: 0.5540 - val_accuracy: 0.7981
Epoch 16/80
140/140 [==============================] - 34s 238ms/step - loss: 0.5130 - accuracy: 0.8055 - val_loss: 0.5636 - val_accuracy: 0.7929
Epoch 17/80
140/140 [==============================] - 35s 242ms/step - loss: 0.4895 - accuracy: 0.8232 - val_loss: 0.5840 - val_accuracy: 0.7772
Epoch 18/80
140/140 [==============================] - 34s 235ms/step - loss: 0.4902 - accuracy: 0.8140 - val_loss: 0.5485 - val_accuracy: 0.8054
Epoch 19/80
140/140 [==============================] - 36s 249ms/step - loss: 0.4817 - accuracy: 0.8255 - val_loss: 0.5405 - val_accuracy: 0.8091
Epoch 20/80
140/140 [==============================] - 38s 267ms/step - loss: 0.4732 - accuracy: 0.8241 - val_loss: 0.5420 - val_accuracy: 0.8070
Epoch 21/80
140/140 [==============================] - 35s 244ms/step - loss: 0.4615 - accuracy: 0.8295 - val_loss: 0.5348 - val_accuracy: 0.8070
Epoch 22/80
140/140 [==============================] - 36s 249ms/step - loss: 0.4496 - accuracy: 0.8326 - val_loss: 0.5191 - val_accuracy: 0.8122
Epoch 23/80
140/140 [==============================] - 35s 245ms/step - loss: 0.4553 - accuracy: 0.8380 - val_loss: 0.5549 - val_accuracy: 0.7882
Epoch 24/80
140/140 [==============================] - 35s 242ms/step - loss: 0.4366 - accuracy: 0.8402 - val_loss: 0.5289 - val_accuracy: 0.8112
Epoch 25/80
140/140 [==============================] - 36s 251ms/step - loss: 0.4471 - accuracy: 0.8335 - val_loss: 0.5254 - val_accuracy: 0.8054
Epoch 26/80
140/140 [==============================] - 34s 238ms/step - loss: 0.4370 - accuracy: 0.8411 - val_loss: 0.5119 - val_accuracy: 0.8154
Epoch 27/80
140/140 [==============================] - 36s 246ms/step - loss: 0.4259 - accuracy: 0.8461 - val_loss: 0.5027 - val_accuracy: 0.8243
Epoch 28/80
140/140 [==============================] - 37s 262ms/step - loss: 0.4207 - accuracy: 0.8463 - val_loss: 0.5160 - val_accuracy: 0.8190
Epoch 29/80
140/140 [==============================] - 35s 241ms/step - loss: 0.4140 - accuracy: 0.8537 - val_loss: 0.4951 - val_accuracy: 0.8217
Epoch 30/80
140/140 [==============================] - 34s 240ms/step - loss: 0.4168 - accuracy: 0.8505 - val_loss: 0.5036 - val_accuracy: 0.8217
Epoch 31/80
140/140 [==============================] - 36s 251ms/step - loss: 0.3959 - accuracy: 0.8606 - val_loss: 0.4913 - val_accuracy: 0.8248
Epoch 32/80
140/140 [==============================] - 35s 246ms/step - loss: 0.3975 - accuracy: 0.8532 - val_loss: 0.4975 - val_accuracy: 0.8237
Epoch 33/80
140/140 [==============================] - 36s 248ms/step - loss: 0.3939 - accuracy: 0.8624 - val_loss: 0.4931 - val_accuracy: 0.8285
Epoch 34/80
140/140 [==============================] - 35s 243ms/step - loss: 0.3915 - accuracy: 0.8597 - val_loss: 0.4893 - val_accuracy: 0.8222
Epoch 35/80
140/140 [==============================] - 36s 245ms/step - loss: 0.3895 - accuracy: 0.8584 - val_loss: 0.5034 - val_accuracy: 0.8154
Epoch 36/80
140/140 [==============================] - 36s 250ms/step - loss: 0.3789 - accuracy: 0.8687 - val_loss: 0.5203 - val_accuracy: 0.8075
Epoch 37/80
140/140 [==============================] - 34s 240ms/step - loss: 0.3770 - accuracy: 0.8669 - val_loss: 0.4891 - val_accuracy: 0.8253
Epoch 38/80
140/140 [==============================] - 36s 247ms/step - loss: 0.3764 - accuracy: 0.8644 - val_loss: 0.4931 - val_accuracy: 0.8232
Epoch 39/80
140/140 [==============================] - 35s 243ms/step - loss: 0.3739 - accuracy: 0.8671 - val_loss: 0.4883 - val_accuracy: 0.8248
Epoch 40/80
140/140 [==============================] - 34s 238ms/step - loss: 0.3686 - accuracy: 0.8707 - val_loss: 0.5011 - val_accuracy: 0.8222
Epoch 41/80
140/140 [==============================] - 34s 239ms/step - loss: 0.3616 - accuracy: 0.8750 - val_loss: 0.4890 - val_accuracy: 0.8300
Epoch 42/80
140/140 [==============================] - 35s 244ms/step - loss: 0.3577 - accuracy: 0.8721 - val_loss: 0.4674 - val_accuracy: 0.8332
Epoch 43/80
140/140 [==============================] - 35s 241ms/step - loss: 0.3538 - accuracy: 0.8792 - val_loss: 0.4744 - val_accuracy: 0.8321
Epoch 44/80
140/140 [==============================] - 35s 244ms/step - loss: 0.3509 - accuracy: 0.8806 - val_loss: 0.4856 - val_accuracy: 0.8285
Epoch 45/80
140/140 [==============================] - 36s 250ms/step - loss: 0.3555 - accuracy: 0.8754 - val_loss: 0.4881 - val_accuracy: 0.8227
Epoch 46/80
140/140 [==============================] - 36s 249ms/step - loss: 0.3502 - accuracy: 0.8790 - val_loss: 0.4835 - val_accuracy: 0.8358
Epoch 47/80
140/140 [==============================] - 34s 240ms/step - loss: 0.3463 - accuracy: 0.8806 - val_loss: 0.4675 - val_accuracy: 0.8410
Epoch 48/80
140/140 [==============================] - 36s 249ms/step - loss: 0.3389 - accuracy: 0.8806 - val_loss: 0.4813 - val_accuracy: 0.8373
Epoch 49/80
140/140 [==============================] - 34s 237ms/step - loss: 0.3465 - accuracy: 0.8808 - val_loss: 0.4679 - val_accuracy: 0.8415
Epoch 50/80
140/140 [==============================] - 39s 269ms/step - loss: 0.3481 - accuracy: 0.8752 - val_loss: 0.4673 - val_accuracy: 0.8394
Epoch 51/80
140/140 [==============================] - 36s 250ms/step - loss: 0.3343 - accuracy: 0.8844 - val_loss: 0.4704 - val_accuracy: 0.8347
Epoch 52/80
140/140 [==============================] - 36s 252ms/step - loss: 0.3380 - accuracy: 0.8848 - val_loss: 0.4633 - val_accuracy: 0.8426
Epoch 53/80
140/140 [==============================] - 36s 245ms/step - loss: 0.3380 - accuracy: 0.8886 - val_loss: 0.4747 - val_accuracy: 0.8353
Epoch 54/80
140/140 [==============================] - 36s 247ms/step - loss: 0.3327 - accuracy: 0.8866 - val_loss: 0.4745 - val_accuracy: 0.8353
Epoch 55/80
140/140 [==============================] - 38s 269ms/step - loss: 0.3261 - accuracy: 0.8884 - val_loss: 0.4759 - val_accuracy: 0.8332
Epoch 56/80
140/140 [==============================] - 38s 269ms/step - loss: 0.3206 - accuracy: 0.8976 - val_loss: 0.4724 - val_accuracy: 0.8358
Epoch 57/80
140/140 [==============================] - 35s 243ms/step - loss: 0.3198 - accuracy: 0.8940 - val_loss: 0.4770 - val_accuracy: 0.8337
Epoch 58/80
140/140 [==============================] - 36s 245ms/step - loss: 0.3214 - accuracy: 0.8893 - val_loss: 0.4731 - val_accuracy: 0.8373
Epoch 59/80
140/140 [==============================] - 35s 243ms/step - loss: 0.3189 - accuracy: 0.8884 - val_loss: 0.4661 - val_accuracy: 0.8452
Epoch 60/80
140/140 [==============================] - 34s 240ms/step - loss: 0.3181 - accuracy: 0.8978 - val_loss: 0.4677 - val_accuracy: 0.8337
Epoch 61/80
140/140 [==============================] - 36s 252ms/step - loss: 0.3250 - accuracy: 0.8882 - val_loss: 0.4635 - val_accuracy: 0.8379
Epoch 62/80
140/140 [==============================] - 34s 239ms/step - loss: 0.3121 - accuracy: 0.8942 - val_loss: 0.4678 - val_accuracy: 0.8368
In [ ]:
# Visualization of the learning curves

plt.subplot(1, 2, 1)
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.ylim([0.0, 2.0])
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.legend(['train', 'valid'])

plt.subplot(1, 2, 2)
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.ylim([0.5, 1.0])
plt.xlabel('Epoch')
plt.ylabel('Accuracy')
plt.legend(['train', 'valid'])

<matplotlib.legend.Legend at 0x7a82c46f8340>
In [ ]:
net.save("/content/gdrive/MyDrive/Deep Learning Project/Segmentedv5.keras")
In [ ]:
net.evaluate(test_ds)

6/6 [==============================] - 372s 64s/step - loss: 0.5476 - accuracy: 0.8079

[0.5475756525993347, 0.807901918888092]
In [ ]:
net.evaluate(test_ds_cropped)

21/21 [==============================] - 1202s 57s/step - loss: 0.5768 - accuracy: 0.8108

[0.5767860412597656, 0.810800313949585]
In [ ]:
net.evaluate(test_ds_cropped_balanced)

6/6 [==============================] - 302s 50s/step - loss: 0.4618 - accuracy: 0.8306

[0.4618443548679352, 0.8306010961532593]

None of the three accuracy resulted to be higher than the test accuracy we achieved with the same model applied to unsegmented data (84.74%). Also, the balancing of the dataset did not seem to increase the performance of the model, since the performances decreased on all the test sets.

Conclusion¶

The model that achieved the best performances remains the one we trained on the unsegmented data. Neither segmentation nor balancing the dataset were able to increase the performances on the test set.